You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "bobhan1 (via GitHub)" <gi...@apache.org> on 2024/09/19 08:27:03 UTC
[PR] [branch-2.0] Picks "[opt](merge-on-write) Reduce the version not continuous logs for merge-on-write table #40946" [doris]
bobhan1 opened a new pull request, #40997:
URL: https://github.com/apache/doris/pull/40997
picks https://github.com/apache/doris/pull/40946
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [branch-2.0] Picks "[opt](merge-on-write) Reduce the version not continuous logs for merge-on-write table #40946" [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40997:
URL: https://github.com/apache/doris/pull/40997#issuecomment-2360525933
<details>
<summary>ClickBench: <b>Total hot run time: 31.29 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1dc16ec281a2ce7d86c87cd968ee7173a405942b, data reload: false
query1 0.03 0.03 0.02
query2 0.07 0.03 0.03
query3 0.24 0.04 0.05
query4 1.82 0.07 0.07
query5 0.54 0.53 0.52
query6 1.25 0.65 0.60
query7 0.01 0.01 0.02
query8 0.05 0.03 0.02
query9 0.53 0.49 0.48
query10 0.53 0.54 0.54
query11 0.12 0.09 0.09
query12 0.11 0.09 0.09
query13 0.61 0.61 0.62
query14 0.78 0.78 0.79
query15 0.78 0.76 0.78
query16 0.38 0.40 0.38
query17 0.99 1.03 1.03
query18 0.25 0.24 0.23
query19 1.91 1.84 1.87
query20 0.02 0.01 0.01
query21 15.45 0.63 0.55
query22 2.05 2.38 2.08
query23 17.48 1.09 0.79
query24 5.88 1.13 1.04
query25 0.36 0.08 0.06
query26 0.68 0.15 0.15
query27 0.04 0.05 0.03
query28 7.09 0.74 0.71
query29 12.71 2.39 2.35
query30 0.57 0.53 0.59
query31 2.81 0.38 0.38
query32 3.38 0.52 0.49
query33 3.08 3.08 3.08
query34 15.28 4.81 4.79
query35 4.87 4.84 4.85
query36 1.08 1.03 1.01
query37 0.06 0.05 0.05
query38 0.03 0.02 0.02
query39 0.02 0.02 0.01
query40 0.16 0.14 0.15
query41 0.06 0.02 0.01
query42 0.02 0.01 0.02
query43 0.02 0.02 0.02
Total cold run time: 104.2 s
Total hot run time: 31.29 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [branch-2.0] Picks "[opt](merge-on-write) Reduce the version not continuous logs for merge-on-write table #40946" [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40997:
URL: https://github.com/apache/doris/pull/40997#issuecomment-2360366192
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [branch-2.0] Picks "[opt](merge-on-write) Reduce the version not continuous logs for merge-on-write table #40946" [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40997:
URL: https://github.com/apache/doris/pull/40997#issuecomment-2360337252
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [branch-2.0] Picks "[opt](merge-on-write) Reduce the version not continuous logs for merge-on-write table #40946" [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #40997:
URL: https://github.com/apache/doris/pull/40997
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [branch-2.0] Picks "[opt](merge-on-write) Reduce the version not continuous logs for merge-on-write table #40946" [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40997:
URL: https://github.com/apache/doris/pull/40997#issuecomment-2360527645
TeamCity be ut coverage result:
Function Coverage: 37.84% (8141/21513)
Line Coverage: 29.59% (66998/226423)
Region Coverage: 29.08% (34566/118884)
Branch Coverage: 24.98% (17810/71294)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1dc16ec281a2ce7d86c87cd968ee7173a405942b_1dc16ec281a2ce7d86c87cd968ee7173a405942b/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [branch-2.0] Picks "[opt](merge-on-write) Reduce the version not continuous logs for merge-on-write table #40946" [doris]
Posted by "bobhan1 (via GitHub)" <gi...@apache.org>.
bobhan1 commented on PR #40997:
URL: https://github.com/apache/doris/pull/40997#issuecomment-2360339602
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [branch-2.0] Picks "[opt](merge-on-write) Reduce the version not continuous logs for merge-on-write table #40946" [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40997:
URL: https://github.com/apache/doris/pull/40997#issuecomment-2360514520
<details>
<summary>TPC-DS: <b>Total hot run time: 211577 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1dc16ec281a2ce7d86c87cd968ee7173a405942b, data reload: false
query1 934 387 436 387
query2 6533 2168 2116 2116
query3 6921 207 200 200
query4 22944 21713 21433 21433
query5 19734 6522 6559 6522
query6 285 224 240 224
query7 4327 299 296 296
query8 269 259 252 252
query9 3117 2692 2620 2620
query10 461 322 322 322
query11 16137 14915 14976 14915
query12 130 72 79 72
query13 1039 448 454 448
query14 17972 13704 13178 13178
query15 365 222 225 222
query16 6486 282 266 266
query17 1914 943 919 919
query18 897 321 311 311
query19 208 152 148 148
query20 82 75 80 75
query21 194 97 96 96
query22 5195 5034 4941 4941
query23 34418 33511 33497 33497
query24 6882 6281 6270 6270
query25 547 435 435 435
query26 806 167 161 161
query27 2254 300 295 295
query28 6118 2307 2244 2244
query29 2926 2826 2815 2815
query30 239 169 168 168
query31 965 739 735 735
query32 71 65 56 56
query33 445 274 273 273
query34 856 487 480 480
query35 1128 933 968 933
query36 1217 1244 1152 1152
query37 92 59 62 59
query38 3034 2948 2914 2914
query39 1380 1325 1314 1314
query40 210 97 102 97
query41 40 43 37 37
query42 93 86 85 85
query43 627 680 565 565
query44 1185 712 719 712
query45 243 233 227 227
query46 1234 959 965 959
query47 1911 1694 1847 1694
query48 515 421 403 403
query49 615 371 391 371
query50 867 627 624 624
query51 4766 4680 4639 4639
query52 89 79 72 72
query53 233 187 182 182
query54 2665 2473 2464 2464
query55 86 81 85 81
query56 214 208 201 201
query57 1259 1155 1173 1155
query58 224 220 213 213
query59 3476 3254 3415 3254
query60 222 209 209 209
query61 100 95 94 94
query62 833 479 514 479
query63 215 179 176 176
query64 3225 1609 1519 1519
query65 3611 3513 3562 3513
query66 727 414 425 414
query67 15825 14852 16646 14852
query68 9629 655 648 648
query69 512 283 292 283
query70 1524 1474 1291 1291
query71 414 318 321 318
query72 6836 4625 4957 4625
query73 755 321 326 321
query74 6368 5839 5923 5839
query75 5323 3714 3685 3685
query76 5487 1111 1189 1111
query77 940 257 259 257
query78 12471 11783 11608 11608
query79 8020 626 638 626
query80 1566 392 385 385
query81 507 243 244 243
query82 1642 98 98 98
query83 164 131 133 131
query84 253 72 70 70
query85 904 325 310 310
query86 341 290 294 290
query87 3258 2973 3026 2973
query88 4712 2298 2292 2292
query89 451 269 278 269
query90 1968 212 220 212
query91 165 130 139 130
query92 54 50 52 50
query93 6310 581 608 581
query94 696 212 208 208
query95 1973 1926 2074 1926
query96 642 326 324 324
query97 6542 6372 6423 6372
query98 219 198 201 198
query99 3029 848 926 848
Total cold run time: 318507 ms
Total hot run time: 211577 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [branch-2.0] Picks "[opt](merge-on-write) Reduce the version not continuous logs for merge-on-write table #40946" [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40997:
URL: https://github.com/apache/doris/pull/40997#issuecomment-2360532585
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 1dc16ec281a2ce7d86c87cd968ee7173a405942b with default session variables
Stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 21.8 seconds inserted 10000000 Rows, about 458K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [branch-2.0] Picks "[opt](merge-on-write) Reduce the version not continuous logs for merge-on-write table #40946" [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40997:
URL: https://github.com/apache/doris/pull/40997#issuecomment-2360487278
<details>
<summary>TPC-H: <b>Total hot run time: 49283 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1dc16ec281a2ce7d86c87cd968ee7173a405942b, data reload: false
------ Round 1 ----------------------------------
q1 17716 4413 4373 4373
q2 2077 159 144 144
q3 10282 1942 1910 1910
q4 10404 1253 1324 1253
q5 8608 3871 3957 3871
q6 235 125 124 124
q7 2071 1631 1600 1600
q8 9500 2787 2748 2748
q9 10428 9837 9932 9837
q10 8645 3543 3536 3536
q11 413 250 251 250
q12 472 302 294 294
q13 18320 3982 4007 3982
q14 363 331 335 331
q15 519 446 453 446
q16 540 474 458 458
q17 1150 953 971 953
q18 7370 6930 6851 6851
q19 1682 1569 1477 1477
q20 546 317 313 313
q21 4498 4159 4136 4136
q22 486 400 396 396
Total cold run time: 116325 ms
Total hot run time: 49283 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4331 4327 4372 4327
q2 329 225 226 225
q3 4202 4145 4123 4123
q4 2764 2767 2746 2746
q5 7162 7115 7085 7085
q6 242 119 120 119
q7 3227 2868 2846 2846
q8 4382 4485 4551 4485
q9 13537 13625 13510 13510
q10 4279 4293 4222 4222
q11 736 675 685 675
q12 1020 833 827 827
q13 6738 3761 3705 3705
q14 457 424 411 411
q15 505 459 449 449
q16 643 607 606 606
q17 3782 3874 3857 3857
q18 8831 8852 8750 8750
q19 1718 1658 1639 1639
q20 2435 2071 2131 2071
q21 8576 8560 8484 8484
q22 1067 934 956 934
Total cold run time: 80963 ms
Total hot run time: 76096 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org