You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "w41ter (via GitHub)" <gi...@apache.org> on 2024/09/09 09:48:32 UTC
[PR] [chore](persist) reduce truncate table operation log size #40422 [doris]
w41ter opened a new pull request, #40554:
URL: https://github.com/apache/doris/pull/40554
cherry pick from #40422
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](persist) reduce truncate table operation log size #40422 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40554:
URL: https://github.com/apache/doris/pull/40554#issuecomment-2337737860
<details>
<summary>ClickBench: <b>Total hot run time: 31.15 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 631facb21135f0bf6b82c52cd914d235e308a1f7, data reload: false
query1 0.02 0.02 0.02
query2 0.07 0.03 0.03
query3 0.24 0.05 0.05
query4 1.82 0.06 0.06
query5 0.54 0.53 0.52
query6 1.22 0.61 0.62
query7 0.02 0.01 0.01
query8 0.03 0.02 0.03
query9 0.52 0.49 0.49
query10 0.54 0.53 0.54
query11 0.12 0.09 0.09
query12 0.12 0.09 0.09
query13 0.63 0.61 0.62
query14 0.80 0.80 0.77
query15 0.79 0.78 0.77
query16 0.37 0.39 0.37
query17 1.00 1.02 1.03
query18 0.21 0.26 0.25
query19 1.86 1.85 1.84
query20 0.02 0.01 0.01
query21 15.49 0.60 0.55
query22 2.17 2.39 1.45
query23 16.85 1.07 1.10
query24 3.83 2.66 1.35
query25 0.34 0.13 0.05
query26 0.64 0.17 0.16
query27 0.04 0.04 0.04
query28 7.28 0.76 0.73
query29 12.58 2.29 2.32
query30 0.59 0.54 0.52
query31 2.81 0.40 0.37
query32 3.37 0.50 0.48
query33 3.09 3.07 3.05
query34 15.26 4.83 4.80
query35 4.87 4.84 4.85
query36 1.07 1.00 1.02
query37 0.06 0.05 0.04
query38 0.04 0.03 0.02
query39 0.02 0.02 0.01
query40 0.16 0.13 0.14
query41 0.06 0.01 0.02
query42 0.03 0.01 0.01
query43 0.02 0.02 0.02
Total cold run time: 101.61 s
Total hot run time: 31.15 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](persist) reduce truncate table operation log size #40422 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40554:
URL: https://github.com/apache/doris/pull/40554#issuecomment-2337650517
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](persist) reduce truncate table operation log size #40422 [doris]
Posted by "w41ter (via GitHub)" <gi...@apache.org>.
w41ter commented on PR #40554:
URL: https://github.com/apache/doris/pull/40554#issuecomment-2337650362
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](persist) reduce truncate table operation log size #40422 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40554:
URL: https://github.com/apache/doris/pull/40554#issuecomment-2337727470
<details>
<summary>TPC-DS: <b>Total hot run time: 211602 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 631facb21135f0bf6b82c52cd914d235e308a1f7, data reload: false
query1 932 428 381 381
query2 6528 2417 2167 2167
query3 6921 212 214 212
query4 23486 21725 21595 21595
query5 19726 6512 6470 6470
query6 287 219 239 219
query7 4343 294 307 294
query8 253 254 239 239
query9 3051 2602 2543 2543
query10 470 314 293 293
query11 15982 14989 14837 14837
query12 133 77 72 72
query13 1023 432 436 432
query14 17676 13702 13580 13580
query15 368 220 229 220
query16 6383 283 257 257
query17 1743 933 906 906
query18 893 316 340 316
query19 219 153 152 152
query20 78 78 79 78
query21 195 94 100 94
query22 5151 5003 4959 4959
query23 34267 33364 33654 33364
query24 6996 6353 6304 6304
query25 526 445 421 421
query26 819 159 161 159
query27 2313 291 293 291
query28 6076 2250 2204 2204
query29 2956 2716 2777 2716
query30 249 167 163 163
query31 965 746 740 740
query32 71 64 60 60
query33 437 267 247 247
query34 865 477 461 461
query35 1121 954 948 948
query36 1389 1075 1228 1075
query37 90 60 60 60
query38 3085 2903 2909 2903
query39 1403 1332 1318 1318
query40 209 95 93 93
query41 39 38 38 38
query42 89 79 91 79
query43 757 605 638 605
query44 1178 714 723 714
query45 240 233 228 228
query46 1241 984 957 957
query47 1769 1852 1733 1733
query48 507 397 393 393
query49 623 385 381 381
query50 873 618 637 618
query51 4755 4742 4717 4717
query52 88 85 84 84
query53 236 181 190 181
query54 2695 2484 2500 2484
query55 94 86 85 85
query56 229 216 210 210
query57 1298 1089 1047 1047
query58 219 212 221 212
query59 3617 3284 3151 3151
query60 209 212 199 199
query61 99 92 94 92
query62 837 473 504 473
query63 200 187 188 187
query64 3316 1557 1505 1505
query65 3613 3545 3560 3545
query66 789 440 418 418
query67 17333 15535 14588 14588
query68 11084 630 632 630
query69 497 277 268 268
query70 1911 1523 1440 1440
query71 424 311 316 311
query72 7026 4792 4756 4756
query73 777 324 318 318
query74 6360 5924 5829 5829
query75 5488 3766 3646 3646
query76 6524 1130 1243 1130
query77 1116 270 264 264
query78 12486 11726 11592 11592
query79 7341 634 633 633
query80 1348 391 384 384
query81 492 244 249 244
query82 1608 98 95 95
query83 164 138 134 134
query84 258 70 70 70
query85 891 316 313 313
query86 345 293 292 292
query87 3212 3045 3032 3032
query88 4361 2385 2384 2384
query89 484 300 288 288
query90 1989 218 210 210
query91 155 127 129 127
query92 61 51 50 50
query93 7085 571 555 555
query94 702 208 213 208
query95 1949 2091 2001 2001
query96 663 341 327 327
query97 6448 6273 6378 6273
query98 221 208 202 202
query99 3014 890 829 829
Total cold run time: 323105 ms
Total hot run time: 211602 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](persist) reduce truncate table operation log size #40422 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40554:
URL: https://github.com/apache/doris/pull/40554#issuecomment-2337743889
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 631facb21135f0bf6b82c52cd914d235e308a1f7 with default session variables
Stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 21.7 seconds inserted 10000000 Rows, about 460K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](persist) reduce truncate table operation log size #40422 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40554:
URL: https://github.com/apache/doris/pull/40554#issuecomment-2337693905
<details>
<summary>TPC-H: <b>Total hot run time: 49880 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 631facb21135f0bf6b82c52cd914d235e308a1f7, data reload: false
------ Round 1 ----------------------------------
q1 17544 4416 4297 4297
q2 2034 192 145 145
q3 10221 1907 1949 1907
q4 10105 1243 1295 1243
q5 8563 3971 3928 3928
q6 231 125 122 122
q7 2030 1596 1622 1596
q8 9337 2772 2736 2736
q9 10213 10424 10400 10400
q10 8654 3567 3504 3504
q11 418 273 251 251
q12 479 298 297 297
q13 18343 4004 4051 4004
q14 360 332 334 332
q15 515 454 466 454
q16 551 455 446 446
q17 1159 936 919 919
q18 7367 6968 7015 6968
q19 1702 1594 1512 1512
q20 551 328 290 290
q21 4435 4172 4139 4139
q22 502 390 401 390
Total cold run time: 115314 ms
Total hot run time: 49880 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4355 4326 4470 4326
q2 325 232 225 225
q3 4240 4202 4177 4177
q4 2780 2744 2739 2739
q5 7139 7144 7143 7143
q6 238 118 118 118
q7 3235 2856 2833 2833
q8 4393 4449 4578 4449
q9 13633 13663 13645 13645
q10 4259 4256 4287 4256
q11 730 695 672 672
q12 1011 838 867 838
q13 4847 3730 3760 3730
q14 458 419 420 419
q15 508 463 458 458
q16 622 597 575 575
q17 3882 3839 3805 3805
q18 8781 8737 8853 8737
q19 1726 1674 1690 1674
q20 2408 2121 2097 2097
q21 8501 8413 8521 8413
q22 1066 928 936 928
Total cold run time: 79137 ms
Total hot run time: 76257 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [chore](persist) reduce truncate table operation log size #40422 [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #40554:
URL: https://github.com/apache/doris/pull/40554
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org