You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "zclllyybb (via GitHub)" <gi...@apache.org> on 2024/07/18 17:32:04 UTC
[PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
zclllyybb opened a new pull request, #38103:
URL: https://github.com/apache/doris/pull/38103
## Proposed changes
Issue Number: close #xxx
Before, if insert overwrite auto detect failed because of some transaction conflicts, it will go into an unexpected situation with some of the partition replace success but some retains.
Now it will wholly success, or wholly failed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2248415732
<details>
<summary>ClickBench: <b>Total hot run time: 30.87 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 74b2aa1793eec78419807ab426c420f8dc7c7659, data reload: false
query1 0.04 0.03 0.03
query2 0.09 0.04 0.04
query3 0.22 0.05 0.06
query4 1.68 0.08 0.07
query5 0.50 0.50 0.49
query6 1.13 0.72 0.72
query7 0.02 0.02 0.02
query8 0.05 0.04 0.04
query9 0.55 0.47 0.50
query10 0.54 0.53 0.55
query11 0.15 0.11 0.12
query12 0.15 0.12 0.12
query13 0.59 0.59 0.58
query14 0.76 0.77 0.78
query15 0.87 0.82 0.82
query16 0.35 0.38 0.37
query17 0.99 1.04 0.98
query18 0.22 0.22 0.21
query19 1.85 1.85 1.81
query20 0.02 0.01 0.01
query21 15.40 0.77 0.65
query22 3.83 6.94 2.15
query23 18.30 1.39 1.24
query24 2.11 0.22 0.21
query25 0.16 0.08 0.08
query26 0.30 0.20 0.21
query27 0.45 0.24 0.23
query28 13.24 1.02 1.00
query29 12.59 3.36 3.30
query30 0.25 0.06 0.05
query31 2.87 0.38 0.39
query32 3.28 0.48 0.47
query33 2.91 2.92 2.90
query34 16.99 4.32 4.27
query35 4.42 4.41 4.42
query36 0.66 0.48 0.48
query37 0.20 0.16 0.16
query38 0.16 0.16 0.16
query39 0.04 0.04 0.04
query40 0.15 0.13 0.12
query41 0.10 0.04 0.05
query42 0.06 0.05 0.06
query43 0.04 0.04 0.04
Total cold run time: 109.28 s
Total hot run time: 30.87 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 merged PR #38103:
URL: https://github.com/apache/doris/pull/38103
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2237135471
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2244085245
<details>
<summary>TPC-DS: <b>Total hot run time: 173961 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2ac1d966834c6309d4f260f2f7c390e8cdfd4ab7, data reload: false
query1 919 381 361 361
query2 6443 1912 1892 1892
query3 6650 212 214 212
query4 23881 17742 17515 17515
query5 3678 476 473 473
query6 280 164 155 155
query7 4594 286 290 286
query8 234 193 195 193
query9 8515 2430 2422 2422
query10 444 292 290 290
query11 12315 10074 10089 10074
query12 118 82 84 82
query13 1640 370 360 360
query14 10213 7793 7629 7629
query15 252 159 164 159
query16 7744 490 479 479
query17 1572 541 538 538
query18 1637 276 281 276
query19 192 147 153 147
query20 91 81 83 81
query21 203 123 122 122
query22 4336 4045 3902 3902
query23 33993 33865 33590 33590
query24 10066 2957 2869 2869
query25 574 383 409 383
query26 697 154 147 147
query27 2212 276 278 276
query28 5804 2065 2050 2050
query29 891 652 645 645
query30 270 155 151 151
query31 973 758 765 758
query32 94 52 54 52
query33 685 349 344 344
query34 915 495 485 485
query35 853 746 737 737
query36 1150 927 1010 927
query37 151 82 79 79
query38 2903 2859 2861 2859
query39 908 875 842 842
query40 219 123 121 121
query41 45 47 45 45
query42 109 110 114 110
query43 518 479 474 474
query44 1106 710 720 710
query45 194 159 160 159
query46 1081 729 702 702
query47 1843 1797 1771 1771
query48 383 292 290 290
query49 836 401 412 401
query50 765 384 381 381
query51 6753 6676 6664 6664
query52 104 93 93 93
query53 361 322 285 285
query54 870 442 433 433
query55 76 71 72 71
query56 290 284 274 274
query57 1131 1051 1052 1051
query58 249 239 278 239
query59 2916 2776 2554 2554
query60 302 269 266 266
query61 98 99 95 95
query62 779 664 629 629
query63 323 284 291 284
query64 9091 2289 1775 1775
query65 3176 3102 3105 3102
query66 765 336 331 331
query67 15655 15192 14905 14905
query68 4480 531 524 524
query69 474 348 343 343
query70 1223 1137 1113 1113
query71 382 291 291 291
query72 6954 6262 5462 5462
query73 746 322 319 319
query74 6096 5687 5673 5673
query75 3363 2701 2696 2696
query76 2128 1030 911 911
query77 421 325 314 314
query78 9742 9248 9034 9034
query79 2470 525 511 511
query80 2856 497 503 497
query81 600 224 224 224
query82 843 140 141 140
query83 321 173 175 173
query84 270 94 97 94
query85 1851 461 346 346
query86 476 337 312 312
query87 3274 3076 3168 3076
query88 4008 2333 2343 2333
query89 481 376 373 373
query90 1746 189 188 188
query91 129 115 101 101
query92 60 48 50 48
query93 3189 495 489 489
query94 1037 288 288 288
query95 410 324 309 309
query96 599 271 274 271
query97 3206 2987 3079 2987
query98 215 198 204 198
query99 1581 1266 1293 1266
Total cold run time: 274759 ms
Total hot run time: 173961 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2244105354
<details>
<summary>TPC-H: <b>Total hot run time: 39896 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a75cb1194f51c47e7873567d95952f02f0ff9fe4, data reload: false
------ Round 1 ----------------------------------
q1 18274 4679 4368 4368
q2 2034 191 197 191
q3 10534 1160 1169 1160
q4 10226 887 758 758
q5 7541 2830 2577 2577
q6 221 139 140 139
q7 953 598 604 598
q8 9217 2097 2042 2042
q9 8921 6599 6586 6586
q10 8770 3796 3782 3782
q11 458 240 246 240
q12 394 230 225 225
q13 17746 2980 2975 2975
q14 282 230 235 230
q15 533 479 479 479
q16 488 384 389 384
q17 961 624 731 624
q18 8101 7608 7377 7377
q19 7783 1352 1371 1352
q20 703 320 337 320
q21 4948 3206 3374 3206
q22 347 283 287 283
Total cold run time: 119435 ms
Total hot run time: 39896 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4433 4380 4282 4282
q2 371 266 274 266
q3 3023 2822 2757 2757
q4 1915 1607 1595 1595
q5 5327 5380 5340 5340
q6 221 131 136 131
q7 2109 1780 1758 1758
q8 3203 3375 3304 3304
q9 8500 8443 8525 8443
q10 3922 3710 3712 3710
q11 592 486 501 486
q12 768 607 588 588
q13 16709 3008 2969 2969
q14 307 288 272 272
q15 505 491 470 470
q16 472 418 423 418
q17 1768 1496 1465 1465
q18 7660 7539 7514 7514
q19 1657 1485 1650 1485
q20 2001 1812 1754 1754
q21 4828 4691 4638 4638
q22 591 510 515 510
Total cold run time: 70882 ms
Total hot run time: 54155 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2248319776
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2237194108
<details>
<summary>TPC-H: <b>Total hot run time: 39646 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f8da9677620df10da06aad322e5d4c5b45133bcb, data reload: false
------ Round 1 ----------------------------------
q1 17627 4360 4277 4277
q2 2011 188 182 182
q3 10458 1163 1048 1048
q4 10180 794 851 794
q5 7514 2655 2689 2655
q6 220 139 138 138
q7 952 597 584 584
q8 9215 2042 2077 2042
q9 8870 6507 6579 6507
q10 8842 3801 3746 3746
q11 440 233 234 233
q12 392 235 220 220
q13 19018 2991 2974 2974
q14 277 234 237 234
q15 523 484 489 484
q16 501 384 378 378
q17 954 648 727 648
q18 8006 7443 7389 7389
q19 6602 1480 1385 1385
q20 681 316 339 316
q21 4919 3128 3278 3128
q22 330 286 284 284
Total cold run time: 118532 ms
Total hot run time: 39646 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4417 4240 4281 4240
q2 368 260 258 258
q3 2998 2873 2942 2873
q4 1997 1660 1691 1660
q5 5597 5543 5438 5438
q6 225 130 130 130
q7 2193 1904 1846 1846
q8 3259 3440 3398 3398
q9 8782 8894 8815 8815
q10 4154 3748 3843 3748
q11 592 488 504 488
q12 827 674 639 639
q13 17392 3186 3154 3154
q14 328 301 289 289
q15 516 487 487 487
q16 482 440 433 433
q17 1788 1503 1500 1500
q18 8130 7979 7847 7847
q19 1744 1540 1613 1540
q20 2166 1890 1844 1844
q21 5098 4743 4815 4743
q22 603 496 538 496
Total cold run time: 73656 ms
Total hot run time: 55866 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2237135510
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2237230774
<details>
<summary>ClickBench: <b>Total hot run time: 30.99 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f8da9677620df10da06aad322e5d4c5b45133bcb, data reload: false
query1 0.04 0.03 0.03
query2 0.08 0.03 0.03
query3 0.22 0.05 0.05
query4 1.67 0.09 0.10
query5 0.50 0.50 0.49
query6 1.14 0.74 0.73
query7 0.02 0.02 0.01
query8 0.05 0.04 0.04
query9 0.55 0.48 0.49
query10 0.55 0.55 0.56
query11 0.15 0.11 0.12
query12 0.14 0.12 0.13
query13 0.59 0.59 0.58
query14 0.75 0.78 0.76
query15 0.84 0.81 0.82
query16 0.35 0.37 0.36
query17 0.98 0.95 0.94
query18 0.22 0.21 0.21
query19 1.80 1.68 1.66
query20 0.01 0.02 0.01
query21 15.38 0.74 0.63
query22 4.02 6.76 2.31
query23 18.29 1.36 1.36
query24 2.16 0.22 0.22
query25 0.16 0.09 0.09
query26 0.30 0.21 0.21
query27 0.45 0.24 0.23
query28 13.26 1.01 0.99
query29 12.60 3.33 3.33
query30 0.25 0.06 0.05
query31 2.89 0.38 0.38
query32 3.30 0.47 0.48
query33 2.90 2.91 2.91
query34 17.19 4.35 4.35
query35 4.42 4.42 4.38
query36 0.66 0.46 0.46
query37 0.19 0.15 0.15
query38 0.16 0.14 0.16
query39 0.05 0.04 0.04
query40 0.15 0.12 0.13
query41 0.09 0.04 0.05
query42 0.05 0.06 0.04
query43 0.05 0.04 0.03
Total cold run time: 109.62 s
Total hot run time: 30.99 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2248373462
<details>
<summary>TPC-H: <b>Total hot run time: 40143 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 74b2aa1793eec78419807ab426c420f8dc7c7659, data reload: false
------ Round 1 ----------------------------------
q1 18393 4389 4350 4350
q2 2581 196 193 193
q3 11189 1173 1144 1144
q4 10363 817 804 804
q5 7901 2703 2732 2703
q6 220 140 141 140
q7 989 629 616 616
q8 9538 2058 2070 2058
q9 8733 6506 6582 6506
q10 8748 3784 3796 3784
q11 465 251 247 247
q12 401 223 225 223
q13 17755 2955 2993 2955
q14 273 240 232 232
q15 525 478 508 478
q16 497 386 383 383
q17 967 644 643 643
q18 7935 7531 7420 7420
q19 6944 1395 1429 1395
q20 707 327 313 313
q21 5023 3278 3272 3272
q22 356 306 284 284
Total cold run time: 120503 ms
Total hot run time: 40143 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4343 4264 4291 4264
q2 387 275 260 260
q3 3024 2738 2796 2738
q4 1880 1623 1589 1589
q5 5335 5331 5300 5300
q6 217 131 133 131
q7 2163 1729 1708 1708
q8 3192 3352 3310 3310
q9 8461 8455 8417 8417
q10 3904 3712 3795 3712
q11 588 496 486 486
q12 758 633 610 610
q13 16764 3013 3042 3013
q14 313 300 276 276
q15 519 480 481 480
q16 475 419 414 414
q17 1779 1507 1469 1469
q18 7710 7595 7463 7463
q19 1659 1523 1505 1505
q20 1991 1800 1830 1800
q21 4950 4703 4805 4703
q22 563 498 519 498
Total cold run time: 70975 ms
Total hot run time: 54146 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2237211893
<details>
<summary>TPC-DS: <b>Total hot run time: 173630 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f8da9677620df10da06aad322e5d4c5b45133bcb, data reload: false
query1 922 371 361 361
query2 6406 1867 1794 1794
query3 6668 212 227 212
query4 28201 17609 17326 17326
query5 3656 470 493 470
query6 250 186 155 155
query7 4578 282 291 282
query8 225 192 202 192
query9 8540 2373 2360 2360
query10 439 290 277 277
query11 10502 10185 10097 10097
query12 117 83 86 83
query13 1665 369 366 366
query14 10164 7697 8063 7697
query15 232 165 171 165
query16 7397 317 321 317
query17 1801 596 550 550
query18 1352 297 283 283
query19 209 162 164 162
query20 94 86 81 81
query21 207 132 126 126
query22 4417 4166 3941 3941
query23 33797 33513 33708 33513
query24 10717 2899 2906 2899
query25 594 434 446 434
query26 696 160 155 155
query27 2303 287 279 279
query28 6139 2065 2041 2041
query29 917 667 643 643
query30 253 163 163 163
query31 1005 765 745 745
query32 95 56 55 55
query33 680 330 396 330
query34 912 498 515 498
query35 689 602 602 602
query36 1111 977 1002 977
query37 140 87 78 78
query38 2945 2865 2819 2819
query39 853 814 811 811
query40 195 121 118 118
query41 44 43 43 43
query42 118 100 109 100
query43 539 491 457 457
query44 1089 726 728 726
query45 192 160 163 160
query46 1095 743 720 720
query47 1870 1811 1753 1753
query48 357 298 294 294
query49 820 410 410 410
query50 788 392 394 392
query51 6811 6862 6784 6784
query52 105 92 94 92
query53 361 282 282 282
query54 862 441 437 437
query55 76 75 72 72
query56 288 265 268 265
query57 1141 1037 1031 1031
query58 247 268 238 238
query59 2912 2576 2778 2576
query60 296 282 269 269
query61 94 91 96 91
query62 794 647 639 639
query63 319 288 291 288
query64 9159 2215 7421 2215
query65 3153 3112 3150 3112
query66 741 330 328 328
query67 15745 15041 14914 14914
query68 5029 536 526 526
query69 689 412 342 342
query70 1189 1049 1062 1049
query71 526 279 273 273
query72 8790 5226 5975 5226
query73 768 332 328 328
query74 6228 5714 5676 5676
query75 4120 2705 2689 2689
query76 3592 938 950 938
query77 701 294 303 294
query78 10657 9067 8968 8968
query79 7314 530 521 521
query80 1398 478 487 478
query81 596 229 218 218
query82 1253 140 134 134
query83 395 213 164 164
query84 273 90 89 89
query85 1446 314 303 303
query86 447 322 323 322
query87 3338 3099 3124 3099
query88 5214 2363 2349 2349
query89 504 390 397 390
query90 1931 192 189 189
query91 134 99 105 99
query92 66 48 48 48
query93 5863 496 491 491
query94 1315 211 212 211
query95 413 320 323 320
query96 617 279 275 275
query97 3179 3011 3019 3011
query98 220 249 197 197
query99 1654 1268 1274 1268
Total cold run time: 291958 ms
Total hot run time: 173630 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2238308792
ADD TEST ！
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2244089269
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on code in PR #38103:
URL: https://github.com/apache/doris/pull/38103#discussion_r1689333607
##########
fe/fe-core/src/main/java/org/apache/doris/insertoverwrite/InsertOverwriteManager.java:
##########
@@ -107,44 +107,82 @@ public void registerTaskInGroup(long groupId, long taskId) {
taskGroups.get(groupId).add(taskId);
}
- public List<Long> tryReplacePartitionIds(long groupId, List<Long> oldPartitionIds) {
+ /**
+ * this func should in lock scope of getLock(groupId)
+ *
+ * @param newIds if have replaced, replace with new. otherwise itself.
+ * @return
+ */
+ public boolean tryReplacePartitionIds(long groupId, List<Long> oldPartitionIds, List<Long> newIds) {
Map<Long, Long> relations = partitionPairs.get(groupId);
- List<Long> newIds = new ArrayList<Long>();
- for (Long id : oldPartitionIds) {
+ boolean needReplace = false;
+ for (int i = 0; i < oldPartitionIds.size(); i++) {
+ long id = oldPartitionIds.get(i);
if (relations.containsKey(id)) {
// if we replaced it. then return new one.
newIds.add(relations.get(id));
} else {
- // otherwise itself. we will deal it soon.
- newIds.add(id);
+ newIds.add(oldPartitionIds.get(i));
Review Comment:
well they're same. will fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2247591689
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on code in PR #38103:
URL: https://github.com/apache/doris/pull/38103#discussion_r1689202593
##########
fe/fe-core/src/main/java/org/apache/doris/insertoverwrite/InsertOverwriteManager.java:
##########
@@ -107,44 +107,82 @@ public void registerTaskInGroup(long groupId, long taskId) {
taskGroups.get(groupId).add(taskId);
}
- public List<Long> tryReplacePartitionIds(long groupId, List<Long> oldPartitionIds) {
+ /**
+ * this func should in lock scope of getLock(groupId)
+ *
+ * @param newIds if have replaced, replace with new. otherwise itself.
+ * @return
+ */
+ public boolean tryReplacePartitionIds(long groupId, List<Long> oldPartitionIds, List<Long> newIds) {
Map<Long, Long> relations = partitionPairs.get(groupId);
- List<Long> newIds = new ArrayList<Long>();
- for (Long id : oldPartitionIds) {
+ boolean needReplace = false;
+ for (int i = 0; i < oldPartitionIds.size(); i++) {
+ long id = oldPartitionIds.get(i);
if (relations.containsKey(id)) {
// if we replaced it. then return new one.
newIds.add(relations.get(id));
} else {
- // otherwise itself. we will deal it soon.
- newIds.add(id);
+ newIds.add(oldPartitionIds.get(i));
Review Comment:
what diffent with `id` and `oldPartitionIds.get(i)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2247665835
<details>
<summary>TPC-H: <b>Total hot run time: 39702 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 627ab0a8861b2d396567be0b042eafe35b847df1, data reload: false
------ Round 1 ----------------------------------
q1 17594 4312 4216 4216
q2 2011 194 204 194
q3 10440 1256 1102 1102
q4 10183 811 775 775
q5 7535 2678 2679 2678
q6 220 140 139 139
q7 940 604 590 590
q8 9225 2049 2071 2049
q9 8680 6559 6569 6559
q10 8871 3757 3811 3757
q11 457 237 250 237
q12 541 231 233 231
q13 17759 2989 2972 2972
q14 282 231 242 231
q15 525 472 507 472
q16 493 381 381 381
q17 954 558 698 558
q18 8077 7364 7414 7364
q19 6171 1477 1409 1409
q20 666 327 332 327
q21 4965 3182 3336 3182
q22 359 283 279 279
Total cold run time: 116948 ms
Total hot run time: 39702 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4397 4269 4246 4246
q2 377 265 268 265
q3 2972 2822 2928 2822
q4 1994 1706 1681 1681
q5 5702 5533 5508 5508
q6 224 133 135 133
q7 2180 1886 1916 1886
q8 3272 3385 3399 3385
q9 8769 8757 8936 8757
q10 4078 3874 3786 3786
q11 578 509 521 509
q12 824 651 634 634
q13 16330 3235 3213 3213
q14 324 288 282 282
q15 531 480 505 480
q16 515 435 445 435
q17 1809 1524 1508 1508
q18 8024 8144 7797 7797
q19 1766 1569 1587 1569
q20 2134 1897 1855 1855
q21 8752 4943 4837 4837
q22 559 518 522 518
Total cold run time: 76111 ms
Total hot run time: 56106 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2244076160
<details>
<summary>TPC-H: <b>Total hot run time: 39540 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2ac1d966834c6309d4f260f2f7c390e8cdfd4ab7, data reload: false
------ Round 1 ----------------------------------
q1 17636 4486 4263 4263
q2 2010 206 190 190
q3 10436 1173 1092 1092
q4 10197 793 854 793
q5 7562 2627 2659 2627
q6 218 135 138 135
q7 952 598 604 598
q8 9203 2047 2071 2047
q9 8718 6538 6538 6538
q10 8808 3850 3752 3752
q11 460 233 242 233
q12 445 222 220 220
q13 17772 3024 2993 2993
q14 272 243 250 243
q15 539 489 504 489
q16 493 385 378 378
q17 950 669 706 669
q18 8125 7529 7284 7284
q19 6738 1466 1246 1246
q20 683 330 318 318
q21 4839 3148 3190 3148
q22 350 285 284 284
Total cold run time: 117406 ms
Total hot run time: 39540 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4394 4251 4299 4251
q2 382 265 266 265
q3 2969 2857 2926 2857
q4 2018 1722 1737 1722
q5 5615 5516 5513 5513
q6 230 133 136 133
q7 2193 1892 1874 1874
q8 3275 3400 3441 3400
q9 8853 8772 8927 8772
q10 4083 3839 3803 3803
q11 594 504 533 504
q12 793 633 631 631
q13 16099 3182 3205 3182
q14 310 298 295 295
q15 530 511 489 489
q16 497 429 438 429
q17 1848 1568 1503 1503
q18 8002 7987 7824 7824
q19 1735 1651 1540 1540
q20 2075 1868 1864 1864
q21 5076 4882 4795 4795
q22 595 505 486 486
Total cold run time: 72166 ms
Total hot run time: 56132 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2244089699
<details>
<summary>ClickBench: <b>Total hot run time: 30.53 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2ac1d966834c6309d4f260f2f7c390e8cdfd4ab7, data reload: false
query1 0.04 0.03 0.03
query2 0.08 0.04 0.03
query3 0.23 0.06 0.05
query4 1.68 0.07 0.08
query5 0.49 0.49 0.49
query6 1.14 0.73 0.73
query7 0.02 0.01 0.02
query8 0.05 0.04 0.04
query9 0.57 0.51 0.50
query10 0.55 0.54 0.54
query11 0.15 0.11 0.12
query12 0.15 0.13 0.13
query13 0.60 0.60 0.58
query14 0.78 0.77 0.78
query15 0.86 0.81 0.82
query16 0.36 0.36 0.36
query17 1.06 0.96 0.96
query18 0.23 0.22 0.22
query19 1.80 1.77 1.70
query20 0.01 0.00 0.01
query21 15.40 0.77 0.66
query22 4.63 7.20 1.74
query23 18.21 1.42 1.27
query24 2.15 0.21 0.21
query25 0.16 0.09 0.09
query26 0.30 0.22 0.22
query27 0.45 0.23 0.22
query28 13.33 1.02 1.00
query29 12.60 3.32 3.32
query30 0.28 0.06 0.05
query31 2.88 0.39 0.40
query32 3.26 0.47 0.47
query33 2.93 2.90 2.87
query34 17.03 4.42 4.39
query35 4.44 4.45 4.46
query36 0.64 0.46 0.46
query37 0.19 0.16 0.15
query38 0.15 0.15 0.15
query39 0.04 0.04 0.04
query40 0.16 0.12 0.12
query41 0.09 0.05 0.04
query42 0.06 0.05 0.05
query43 0.04 0.04 0.04
Total cold run time: 110.27 s
Total hot run time: 30.53 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2244047449
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2244114219
<details>
<summary>TPC-DS: <b>Total hot run time: 173042 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a75cb1194f51c47e7873567d95952f02f0ff9fe4, data reload: false
query1 921 375 372 372
query2 6447 1905 1907 1905
query3 6645 205 219 205
query4 28740 17562 17287 17287
query5 4234 481 476 476
query6 286 205 167 167
query7 4605 290 275 275
query8 237 201 193 193
query9 8594 2435 2398 2398
query10 438 285 266 266
query11 10945 10003 10106 10003
query12 127 91 83 83
query13 1622 364 362 362
query14 10219 7765 7577 7577
query15 210 167 166 166
query16 7823 491 470 470
query17 1371 532 510 510
query18 1967 274 277 274
query19 198 150 155 150
query20 89 85 81 81
query21 198 125 122 122
query22 4185 4087 3887 3887
query23 33834 33338 33167 33167
query24 11309 2866 2925 2866
query25 630 364 360 360
query26 1458 146 141 141
query27 3006 267 263 263
query28 7754 1979 1969 1969
query29 1001 618 643 618
query30 279 148 148 148
query31 982 752 746 746
query32 98 54 54 54
query33 765 326 317 317
query34 903 470 474 470
query35 855 739 739 739
query36 1100 920 923 920
query37 146 81 76 76
query38 2829 2720 2712 2712
query39 856 818 798 798
query40 272 119 122 119
query41 46 47 46 46
query42 119 100 102 100
query43 517 469 464 464
query44 1195 713 717 713
query45 190 165 160 160
query46 1076 734 742 734
query47 1875 1773 1776 1773
query48 365 283 286 283
query49 1121 412 419 412
query50 797 412 390 390
query51 6669 6771 6654 6654
query52 111 89 93 89
query53 363 286 284 284
query54 949 435 455 435
query55 76 73 75 73
query56 293 260 264 260
query57 1135 1053 1036 1036
query58 241 244 240 240
query59 2819 2662 2811 2662
query60 299 294 310 294
query61 99 94 95 94
query62 841 643 666 643
query63 320 294 289 289
query64 10609 2261 1656 1656
query65 3196 3109 3093 3093
query66 1404 351 349 349
query67 15571 15167 15041 15041
query68 4577 517 524 517
query69 473 339 326 326
query70 1179 1126 1158 1126
query71 403 281 275 275
query72 7065 5764 5897 5764
query73 740 324 323 323
query74 6160 5718 5601 5601
query75 3408 2661 2713 2661
query76 2462 904 989 904
query77 452 312 317 312
query78 9583 9011 9011 9011
query79 2906 530 512 512
query80 2146 479 464 464
query81 573 223 221 221
query82 757 140 134 134
query83 291 172 179 172
query84 277 90 89 89
query85 2113 307 309 307
query86 497 316 319 316
query87 3302 3133 3087 3087
query88 3962 2372 2380 2372
query89 486 379 377 377
query90 1865 194 194 194
query91 139 113 115 113
query92 63 51 52 51
query93 4176 491 490 490
query94 1303 306 304 304
query95 429 323 330 323
query96 608 271 273 271
query97 3231 2978 3033 2978
query98 226 210 201 201
query99 1683 1290 1267 1267
Total cold run time: 288009 ms
Total hot run time: 173042 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2249916925
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2244119063
<details>
<summary>ClickBench: <b>Total hot run time: 30.59 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a75cb1194f51c47e7873567d95952f02f0ff9fe4, data reload: false
query1 0.04 0.03 0.03
query2 0.07 0.04 0.04
query3 0.22 0.05 0.05
query4 1.68 0.09 0.09
query5 0.49 0.50 0.49
query6 1.12 0.74 0.72
query7 0.02 0.01 0.02
query8 0.05 0.04 0.04
query9 0.54 0.50 0.48
query10 0.55 0.55 0.55
query11 0.16 0.11 0.12
query12 0.15 0.13 0.12
query13 0.60 0.61 0.58
query14 0.77 0.78 0.77
query15 0.86 0.81 0.80
query16 0.36 0.38 0.35
query17 1.05 0.98 0.96
query18 0.21 0.21 0.21
query19 1.80 1.69 1.68
query20 0.02 0.00 0.01
query21 15.39 0.76 0.66
query22 4.05 7.54 1.76
query23 18.28 1.35 1.29
query24 2.11 0.23 0.24
query25 0.16 0.09 0.09
query26 0.29 0.22 0.22
query27 0.46 0.24 0.23
query28 13.29 1.04 1.02
query29 12.69 3.35 3.31
query30 0.25 0.06 0.06
query31 2.87 0.41 0.40
query32 3.25 0.49 0.48
query33 2.85 2.92 2.97
query34 17.05 4.35 4.35
query35 4.51 4.46 4.46
query36 0.65 0.46 0.47
query37 0.19 0.15 0.16
query38 0.17 0.15 0.14
query39 0.04 0.04 0.04
query40 0.15 0.13 0.12
query41 0.09 0.04 0.05
query42 0.06 0.05 0.05
query43 0.04 0.05 0.03
Total cold run time: 109.65 s
Total hot run time: 30.59 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2249916869
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2247700046
<details>
<summary>ClickBench: <b>Total hot run time: 30.96 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 627ab0a8861b2d396567be0b042eafe35b847df1, data reload: false
query1 0.05 0.03 0.03
query2 0.08 0.04 0.04
query3 0.22 0.04 0.04
query4 1.68 0.07 0.07
query5 0.52 0.48 0.48
query6 1.12 0.72 0.73
query7 0.02 0.02 0.01
query8 0.05 0.04 0.05
query9 0.56 0.50 0.48
query10 0.54 0.54 0.56
query11 0.16 0.11 0.12
query12 0.15 0.12 0.12
query13 0.60 0.60 0.58
query14 0.77 0.78 0.77
query15 0.85 0.82 0.81
query16 0.38 0.37 0.36
query17 1.01 0.95 1.02
query18 0.22 0.21 0.22
query19 1.92 1.80 1.81
query20 0.01 0.02 0.02
query21 15.39 0.77 0.64
query22 4.25 6.78 2.18
query23 18.26 1.33 1.28
query24 2.06 0.24 0.25
query25 0.15 0.08 0.08
query26 0.30 0.21 0.21
query27 0.45 0.23 0.23
query28 13.25 1.02 1.00
query29 12.62 3.29 3.28
query30 0.26 0.06 0.06
query31 2.86 0.40 0.39
query32 3.26 0.49 0.48
query33 2.88 2.96 2.95
query34 16.93 4.32 4.45
query35 4.42 4.44 4.38
query36 0.65 0.47 0.48
query37 0.19 0.16 0.16
query38 0.15 0.15 0.15
query39 0.04 0.04 0.03
query40 0.16 0.12 0.12
query41 0.10 0.05 0.05
query42 0.06 0.05 0.04
query43 0.04 0.04 0.04
Total cold run time: 109.64 s
Total hot run time: 30.96 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2247688750
<details>
<summary>TPC-DS: <b>Total hot run time: 174107 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 627ab0a8861b2d396567be0b042eafe35b847df1, data reload: false
query1 912 373 370 370
query2 6448 1886 1822 1822
query3 6651 202 213 202
query4 28105 17638 17370 17370
query5 3658 476 515 476
query6 274 166 164 164
query7 4589 285 283 283
query8 236 195 190 190
query9 8693 2429 2395 2395
query10 439 284 273 273
query11 10802 10073 10085 10073
query12 128 84 85 84
query13 1647 376 376 376
query14 10147 7738 7697 7697
query15 227 172 173 172
query16 7578 518 428 428
query17 1199 534 532 532
query18 1951 274 276 274
query19 187 147 149 147
query20 90 79 81 79
query21 203 123 124 123
query22 4445 4137 4022 4022
query23 34063 34139 33885 33885
query24 11004 2910 2924 2910
query25 575 403 384 384
query26 726 149 149 149
query27 2285 285 287 285
query28 6101 2082 2080 2080
query29 874 656 635 635
query30 250 151 150 150
query31 947 753 774 753
query32 99 55 58 55
query33 647 341 316 316
query34 910 491 492 491
query35 845 772 778 772
query36 1163 979 908 908
query37 141 85 77 77
query38 2951 2888 2845 2845
query39 914 826 824 824
query40 197 122 120 120
query41 45 44 43 43
query42 121 101 98 98
query43 498 444 443 443
query44 1050 720 739 720
query45 206 180 172 172
query46 1093 709 708 708
query47 1857 1787 1777 1777
query48 365 292 281 281
query49 823 409 412 409
query50 763 387 389 387
query51 6797 6669 6678 6669
query52 104 92 91 91
query53 355 286 313 286
query54 885 452 443 443
query55 74 76 73 73
query56 287 263 264 263
query57 1136 1073 1056 1056
query58 244 245 258 245
query59 2755 2574 2653 2574
query60 296 280 283 280
query61 95 91 91 91
query62 775 663 639 639
query63 318 287 285 285
query64 9142 2217 1656 1656
query65 3152 3074 3108 3074
query66 687 338 322 322
query67 15604 15066 14915 14915
query68 4843 546 556 546
query69 612 417 360 360
query70 1160 1158 1089 1089
query71 402 287 276 276
query72 7902 6007 5600 5600
query73 779 325 324 324
query74 6251 5697 5692 5692
query75 3379 2687 2686 2686
query76 2988 911 847 847
query77 611 308 295 295
query78 9976 9374 9023 9023
query79 3207 518 528 518
query80 2129 476 470 470
query81 582 221 218 218
query82 899 141 137 137
query83 311 163 161 161
query84 266 95 82 82
query85 1406 326 308 308
query86 460 305 319 305
query87 3300 3124 3177 3124
query88 4337 2346 2380 2346
query89 491 375 376 375
query90 1806 192 190 190
query91 128 101 99 99
query92 56 49 50 49
query93 4836 516 512 512
query94 1198 300 285 285
query95 402 310 375 310
query96 598 274 274 274
query97 3178 2982 3012 2982
query98 217 194 193 193
query99 1596 1246 1240 1240
Total cold run time: 283134 ms
Total hot run time: 174107 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](insert-overwrite) Fix insert overwrite auto detect transaction safe [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38103:
URL: https://github.com/apache/doris/pull/38103#issuecomment-2248401032
<details>
<summary>TPC-DS: <b>Total hot run time: 173147 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 74b2aa1793eec78419807ab426c420f8dc7c7659, data reload: false
query1 915 374 374 374
query2 6441 1896 1807 1807
query3 6648 203 212 203
query4 22893 17535 17470 17470
query5 4186 460 477 460
query6 277 183 160 160
query7 4587 292 285 285
query8 243 200 205 200
query9 8469 2422 2400 2400
query10 445 306 276 276
query11 10791 10162 10135 10135
query12 136 88 85 85
query13 1659 396 370 370
query14 10169 8016 7644 7644
query15 219 166 164 164
query16 7414 472 443 443
query17 1573 536 521 521
query18 1435 272 271 271
query19 195 147 147 147
query20 86 85 80 80
query21 210 124 145 124
query22 4483 3932 3959 3932
query23 33795 33095 33263 33095
query24 11844 2965 2855 2855
query25 660 367 370 367
query26 1725 150 146 146
query27 2737 266 293 266
query28 7490 2014 1999 1999
query29 1013 619 608 608
query30 286 148 146 146
query31 1024 726 737 726
query32 94 52 52 52
query33 772 320 321 320
query34 908 482 478 478
query35 882 738 725 725
query36 1110 948 963 948
query37 156 85 79 79
query38 2837 2754 2774 2754
query39 890 817 808 808
query40 275 121 120 120
query41 48 46 44 44
query42 114 99 104 99
query43 497 482 456 456
query44 1226 720 732 720
query45 202 176 173 173
query46 1090 731 687 687
query47 1839 1791 1763 1763
query48 381 291 281 281
query49 1166 408 414 408
query50 806 396 391 391
query51 6794 6733 6684 6684
query52 103 87 93 87
query53 354 290 289 289
query54 992 455 442 442
query55 73 72 74 72
query56 296 264 273 264
query57 1153 1051 1068 1051
query58 257 255 257 255
query59 2913 2698 2645 2645
query60 289 278 262 262
query61 97 101 127 101
query62 815 659 659 659
query63 320 303 282 282
query64 10608 2218 1670 1670
query65 3249 3142 3087 3087
query66 1308 341 328 328
query67 15373 15106 14839 14839
query68 4650 540 561 540
query69 538 477 356 356
query70 1210 1155 1119 1119
query71 397 276 271 271
query72 7242 5750 5806 5750
query73 776 330 327 327
query74 6145 5653 5716 5653
query75 3396 2676 2714 2676
query76 2599 881 872 872
query77 492 299 297 297
query78 11756 9314 8953 8953
query79 4254 519 529 519
query80 1127 479 472 472
query81 569 219 221 219
query82 695 133 129 129
query83 196 173 168 168
query84 279 86 84 84
query85 706 310 314 310
query86 468 338 330 330
query87 3288 3076 3081 3076
query88 4711 2385 2383 2383
query89 505 395 389 389
query90 1953 189 194 189
query91 140 112 177 112
query92 69 47 49 47
query93 4280 515 507 507
query94 1132 290 288 288
query95 400 321 307 307
query96 610 269 278 269
query97 3198 3014 2997 2997
query98 232 202 195 195
query99 1622 1283 1261 1261
Total cold run time: 283970 ms
Total hot run time: 173147 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org