You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "hust-hhb (via GitHub)" <gi...@apache.org> on 2024/02/07 04:25:39 UTC
[PR] [improvement](group_commit) Add bvar to monitor the count of replaying wal fail on group commit [doris]
hust-hhb opened a new pull request, #30941:
URL: https://github.com/apache/doris/pull/30941
## Proposed changes
Add bvar to monitor the count of replaying wal fail on group commit
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_commit) Add bvar to monitor the count of replaying wal fail on group commit [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30941:
URL: https://github.com/apache/doris/pull/30941#issuecomment-1931296620
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit cff3a807d7021badbff26c0ae481d2cbeb2d415c with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.6 seconds inserted 10000000 Rows, about 735K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_commit) Add bvar to monitor the count of replaying wal fail on group commit [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30941:
URL: https://github.com/apache/doris/pull/30941#issuecomment-1931294126
<details>
<summary>ClickBench: <b>Total hot run time: 30.92 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cff3a807d7021badbff26c0ae481d2cbeb2d415c, data reload: false
query1 0.04 0.03 0.03
query2 0.06 0.02 0.02
query3 0.22 0.06 0.06
query4 1.68 0.10 0.09
query5 0.52 0.51 0.54
query6 1.18 0.63 0.64
query7 0.02 0.02 0.01
query8 0.04 0.03 0.02
query9 0.54 0.50 0.49
query10 0.57 0.56 0.55
query11 0.12 0.08 0.08
query12 0.11 0.09 0.09
query13 0.59 0.60 0.60
query14 0.78 0.80 0.80
query15 0.79 0.77 0.75
query16 0.39 0.38 0.39
query17 1.05 1.05 1.02
query18 0.21 0.26 0.26
query19 1.83 1.76 1.74
query20 0.01 0.01 0.01
query21 15.40 0.59 0.57
query22 2.74 2.19 1.74
query23 17.43 0.79 0.82
query24 2.55 1.19 0.91
query25 0.36 0.06 0.24
query26 0.54 0.15 0.14
query27 0.05 0.05 0.06
query28 11.45 0.85 0.84
query29 12.59 3.34 3.28
query30 0.69 0.57 0.53
query31 2.78 0.35 0.35
query32 3.38 0.48 0.48
query33 3.19 3.19 3.19
query34 15.82 4.26 4.34
query35 4.32 4.29 4.28
query36 1.10 1.06 1.05
query37 0.06 0.05 0.05
query38 0.04 0.03 0.02
query39 0.02 0.02 0.02
query40 0.18 0.13 0.13
query41 0.07 0.02 0.01
query42 0.02 0.02 0.01
query43 0.02 0.02 0.02
Total cold run time: 105.55 s
Total hot run time: 30.92 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_commit) Add bvar to monitor the count of replaying wal fail on group commit [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30941:
URL: https://github.com/apache/doris/pull/30941#issuecomment-1931322019
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_commit) Add bvar to monitor the count of replaying wal fail on group commit [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30941:
URL: https://github.com/apache/doris/pull/30941#issuecomment-1931289445
<details>
<summary>TPC-DS: <b>Total hot run time: 173806 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cff3a807d7021badbff26c0ae481d2cbeb2d415c, data reload: false
query1 920 356 336 336
query2 6536 2000 1865 1865
query3 6693 210 198 198
query4 31800 21993 21962 21962
query5 4296 362 424 362
query6 263 171 163 163
query7 4599 292 282 282
query8 239 167 174 167
query9 9185 2242 2231 2231
query10 414 206 216 206
query11 18039 15296 15396 15296
query12 127 73 74 73
query13 1618 438 415 415
query14 9302 6997 6545 6545
query15 230 180 182 180
query16 8116 255 255 255
query17 1865 546 507 507
query18 2109 264 259 259
query19 353 135 140 135
query20 84 75 80 75
query21 196 124 122 122
query22 4879 4644 4710 4644
query23 30847 29950 29911 29911
query24 11640 2781 2755 2755
query25 600 352 349 349
query26 1765 146 146 146
query27 2989 308 305 305
query28 7729 1828 1809 1809
query29 969 617 604 604
query30 290 133 139 133
query31 905 689 703 689
query32 94 58 56 56
query33 727 222 214 214
query34 1164 484 472 472
query35 846 752 742 742
query36 1034 925 929 925
query37 255 58 67 58
query38 3266 3082 3119 3082
query39 1304 1257 1242 1242
query40 265 96 97 96
query41 37 35 35 35
query42 94 91 89 89
query43 515 521 490 490
query44 1087 681 688 681
query45 193 173 172 172
query46 1073 662 658 658
query47 1580 1403 1390 1390
query48 452 372 356 356
query49 1224 282 289 282
query50 772 397 385 385
query51 5167 5192 5118 5118
query52 98 86 85 85
query53 331 262 265 262
query54 291 224 224 224
query55 78 78 78 78
query56 216 204 204 204
query57 976 895 904 895
query58 211 182 185 182
query59 2564 2422 2416 2416
query60 228 215 222 215
query61 84 86 81 81
query62 647 358 358 358
query63 295 262 263 262
query64 6416 3718 3654 3654
query65 3235 3239 3205 3205
query66 1349 316 312 312
query67 14364 14042 13983 13983
query68 4289 537 542 537
query69 469 340 330 330
query70 1250 1183 1255 1183
query71 320 239 249 239
query72 5980 2846 2656 2656
query73 714 330 321 321
query74 6609 6295 6170 6170
query75 3009 2308 2283 2283
query76 2536 974 955 955
query77 364 233 222 222
query78 9109 8792 8567 8567
query79 2408 497 494 494
query80 1818 342 343 342
query81 526 193 195 193
query82 775 83 80 80
query83 248 119 125 119
query84 285 81 90 81
query85 2375 335 326 326
query86 477 302 284 284
query87 3426 3241 3254 3241
query88 3838 2370 2356 2356
query89 433 355 357 355
query90 2026 163 160 160
query91 167 122 124 122
query92 56 41 41 41
query93 2457 483 462 462
query94 1329 171 175 171
query95 479 351 7976 351
query96 593 269 264 264
query97 4217 4108 4094 4094
query98 224 200 187 187
query99 1153 714 682 682
Total cold run time: 286088 ms
Total hot run time: 173806 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_commit) Add bvar to monitor the count of replaying wal fail on group commit [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30941:
URL: https://github.com/apache/doris/pull/30941#issuecomment-1931278712
<details>
<summary>TPC-H: <b>Total hot run time: 37242 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cff3a807d7021badbff26c0ae481d2cbeb2d415c, data reload: false
------ Round 1 ----------------------------------
q1 17659 5671 4563 4563
q2 2434 148 135 135
q3 11340 954 930 930
q4 5226 783 744 744
q5 7879 2991 2978 2978
q6 184 125 123 123
q7 1110 751 737 737
q8 9780 2061 2031 2031
q9 7956 6385 6291 6291
q10 8126 2454 2425 2425
q11 417 198 210 198
q12 762 282 271 271
q13 18035 3310 3301 3301
q14 275 251 246 246
q15 539 491 488 488
q16 476 411 407 407
q17 949 484 509 484
q18 6696 6014 5861 5861
q19 1562 1388 1387 1387
q20 574 351 356 351
q21 6948 2982 3075 2982
q22 797 309 318 309
Total cold run time: 109724 ms
Total hot run time: 37242 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4395 4486 4369 4369
q2 326 245 236 236
q3 2962 2819 2757 2757
q4 1899 1655 1605 1605
q5 5155 5187 5206 5187
q6 191 115 114 114
q7 2122 1788 1816 1788
q8 3101 3256 3254 3254
q9 8343 8298 8277 8277
q10 5878 3544 3492 3492
q11 524 456 460 456
q12 736 567 571 567
q13 4942 3103 3089 3089
q14 271 252 255 252
q15 528 499 490 490
q16 512 494 460 460
q17 1866 1656 1670 1656
q18 8068 7704 7578 7578
q19 9975 1524 1506 1506
q20 2130 1920 1892 1892
q21 4844 4570 4724 4570
q22 541 452 456 452
Total cold run time: 69309 ms
Total hot run time: 54047 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_commit) Add bvar to monitor the count of replaying wal fail on group commit [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30941:
URL: https://github.com/apache/doris/pull/30941#issuecomment-1931274665
TeamCity be ut coverage result:
Function Coverage: 36.17% (8638/23882)
Line Coverage: 28.20% (70627/250476)
Region Coverage: 27.18% (36438/134042)
Branch Coverage: 23.97% (18661/77850)
Coverage Report: http://coverage.selectdb-in.cc/coverage/cff3a807d7021badbff26c0ae481d2cbeb2d415c_cff3a807d7021badbff26c0ae481d2cbeb2d415c/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_commit) Add bvar to monitor the count of replaying wal fail on group commit [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30941:
URL: https://github.com/apache/doris/pull/30941#issuecomment-1931258665
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_commit) Add bvar to monitor the count of replaying wal fail on group commit [doris]
Posted by "hust-hhb (via GitHub)" <gi...@apache.org>.
hust-hhb commented on PR #30941:
URL: https://github.com/apache/doris/pull/30941#issuecomment-1931258750
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_commit) Add bvar to monitor the count of replaying wal fail on group commit [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30941:
URL: https://github.com/apache/doris/pull/30941#issuecomment-1931266011
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_commit) Add bvar to monitor the count of replaying wal fail on group commit [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30941:
URL: https://github.com/apache/doris/pull/30941#issuecomment-1931322050
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](group_commit) Add bvar to monitor the count of replaying wal fail on group commit [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #30941:
URL: https://github.com/apache/doris/pull/30941
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org