You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "kaijchen (via GitHub)" <gi...@apache.org> on 2024/01/25 14:13:49 UTC
[PR] [debug](move-memtable) enable brpc debug log in regression pipelines [doris]
kaijchen opened a new pull request, #30389:
URL: https://github.com/apache/doris/pull/30389
## Proposed changes
Enable brpc debug log in regression pipelines, for debugging close_wait timeout.
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](move-memtable) enable brpc debug log in regression pipelines [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30389:
URL: https://github.com/apache/doris/pull/30389#issuecomment-1911798261
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](move-memtable) enable brpc debug log in regression pipelines [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #30389:
URL: https://github.com/apache/doris/pull/30389
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](move-memtable) enable brpc debug log in regression pipelines [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30389:
URL: https://github.com/apache/doris/pull/30389#issuecomment-1911626367
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 6f90d88e94f8418e24bc1045383b83ac245daf9b with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 13.4 seconds inserted 10000000 Rows, about 746K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](move-memtable) enable brpc debug log in regression pipelines [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30389:
URL: https://github.com/apache/doris/pull/30389#issuecomment-1911336685
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](move-memtable) enable brpc debug log in regression pipelines [doris]
Posted by "kaijchen (via GitHub)" <gi...@apache.org>.
kaijchen commented on PR #30389:
URL: https://github.com/apache/doris/pull/30389#issuecomment-1910298390
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](move-memtable) enable brpc debug log in regression pipelines [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30389:
URL: https://github.com/apache/doris/pull/30389#issuecomment-1911623358
<details>
<summary>ClickBench: <b>Total hot run time: 30.43 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6f90d88e94f8418e24bc1045383b83ac245daf9b, data reload: false
query1 0.04 0.02 0.02
query2 0.06 0.02 0.02
query3 0.22 0.04 0.04
query4 1.72 0.07 0.08
query5 0.54 0.53 0.53
query6 1.24 0.63 0.64
query7 0.01 0.02 0.01
query8 0.04 0.02 0.02
query9 0.54 0.49 0.49
query10 0.53 0.54 0.54
query11 0.11 0.09 0.08
query12 0.11 0.09 0.09
query13 0.61 0.62 0.61
query14 0.78 0.80 0.80
query15 0.78 0.76 0.76
query16 0.39 0.39 0.39
query17 1.02 1.01 0.97
query18 0.23 0.23 0.24
query19 1.80 1.78 1.77
query20 0.01 0.01 0.01
query21 15.40 0.59 0.58
query22 2.59 2.37 1.75
query23 17.40 0.71 0.76
query24 2.46 1.22 0.84
query25 0.39 0.16 0.06
query26 0.60 0.12 0.12
query27 0.06 0.06 0.04
query28 11.25 0.76 0.75
query29 12.50 3.26 3.12
query30 0.53 0.57 0.56
query31 2.78 0.33 0.34
query32 3.41 0.48 0.48
query33 3.23 3.26 3.21
query34 15.81 4.23 4.29
query35 4.32 4.26 4.30
query36 1.10 1.05 1.07
query37 0.06 0.05 0.05
query38 0.04 0.03 0.03
query39 0.02 0.01 0.02
query40 0.16 0.13 0.13
query41 0.07 0.02 0.01
query42 0.03 0.01 0.02
query43 0.03 0.02 0.02
Total cold run time: 105.02 s
Total hot run time: 30.43 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](move-memtable) enable brpc debug log in regression pipelines [doris]
Posted by "kaijchen (via GitHub)" <gi...@apache.org>.
kaijchen commented on PR #30389:
URL: https://github.com/apache/doris/pull/30389#issuecomment-1911332911
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](move-memtable) enable brpc debug log in regression pipelines [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30389:
URL: https://github.com/apache/doris/pull/30389#issuecomment-1910308047
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](move-memtable) enable brpc debug log in regression pipelines [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30389:
URL: https://github.com/apache/doris/pull/30389#issuecomment-1911572426
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](move-memtable) enable brpc debug log in regression pipelines [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30389:
URL: https://github.com/apache/doris/pull/30389#issuecomment-1910308124
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](move-memtable) enable brpc debug log in regression pipelines [doris]
Posted by "kaijchen (via GitHub)" <gi...@apache.org>.
kaijchen commented on PR #30389:
URL: https://github.com/apache/doris/pull/30389#issuecomment-1911563132
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](move-memtable) enable brpc debug log in regression pipelines [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30389:
URL: https://github.com/apache/doris/pull/30389#issuecomment-1911608197
<details>
<summary>TPC-H: <b>Total hot run time: 38921 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6f90d88e94f8418e24bc1045383b83ac245daf9b, data reload: false
------ Round 1 ----------------------------------
q1 17637 5234 5475 5234
q2 2047 140 128 128
q3 10644 1198 1204 1198
q4 10224 806 821 806
q5 7731 3119 3192 3119
q6 197 122 121 121
q7 876 516 485 485
q8 9227 1957 1955 1955
q9 7290 6420 6337 6337
q10 8217 3103 3079 3079
q11 404 207 196 196
q12 350 195 192 192
q13 18000 3374 3368 3368
q14 254 220 221 220
q15 541 527 505 505
q16 451 393 394 393
q17 965 530 501 501
q18 7585 6869 6713 6713
q19 1542 1432 1396 1396
q20 557 301 285 285
q21 2741 2393 2460 2393
q22 372 297 324 297
Total cold run time: 107852 ms
Total hot run time: 38921 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5540 5222 5239 5222
q2 329 217 212 212
q3 3331 3248 3163 3163
q4 2087 2104 2024 2024
q5 6086 5876 5929 5876
q6 197 115 119 115
q7 2338 1889 1869 1869
q8 3236 3353 3375 3353
q9 9017 8832 8751 8751
q10 3914 3846 3861 3846
q11 567 456 462 456
q12 795 614 615 614
q13 16920 3153 3173 3153
q14 292 250 272 250
q15 542 504 507 504
q16 506 479 468 468
q17 1898 1826 1863 1826
q18 9457 18347 9752 9752
q19 24449 1593 1520 1520
q20 4606 1962 1924 1924
q21 14449 5303 5303 5303
q22 981 533 555 533
Total cold run time: 111537 ms
Total hot run time: 60734 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](move-memtable) enable brpc debug log in regression pipelines [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30389:
URL: https://github.com/apache/doris/pull/30389#issuecomment-1911618530
<details>
<summary>TPC-DS: <b>Total hot run time: 187428 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6f90d88e94f8418e24bc1045383b83ac245daf9b, data reload: false
query1 923 336 334 334
query2 6576 1998 2108 1998
query3 6729 204 206 204
query4 31641 22194 22126 22126
query5 4455 449 368 368
query6 243 155 147 147
query7 4605 268 258 258
query8 218 168 178 168
query9 8316 2564 2555 2555
query10 422 217 218 217
query11 16634 15539 15525 15525
query12 120 72 65 65
query13 1669 378 382 378
query14 10517 6953 6972 6953
query15 219 183 178 178
query16 5784 257 252 252
query17 942 490 481 481
query18 1778 255 256 255
query19 182 133 129 129
query20 73 75 68 68
query21 189 135 125 125
query22 5178 4872 5067 4872
query23 31648 30849 30762 30762
query24 11415 2767 2810 2767
query25 532 312 292 292
query26 884 141 149 141
query27 2859 282 288 282
query28 6231 1860 1851 1851
query29 929 638 626 626
query30 277 139 133 133
query31 921 730 750 730
query32 84 52 47 47
query33 659 210 215 210
query34 1069 453 466 453
query35 874 784 753 753
query36 1321 1247 1233 1233
query37 91 60 60 60
query38 3357 3277 3225 3225
query39 1293 1271 1252 1252
query40 194 89 78 78
query41 37 35 34 34
query42 84 82 81 81
query43 528 501 479 479
query44 1075 698 707 698
query45 198 172 177 172
query46 1052 631 659 631
query47 1653 1585 1561 1561
query48 387 304 310 304
query49 1093 292 289 289
query50 681 307 306 306
query51 5311 5190 5164 5164
query52 94 79 87 79
query53 325 248 249 248
query54 234 178 186 178
query55 80 74 77 74
query56 177 163 163 163
query57 997 927 951 927
query58 171 158 154 154
query59 2918 2651 2770 2651
query60 214 181 179 179
query61 83 92 83 83
query62 594 372 363 363
query63 292 253 258 253
query64 4299 1796 1793 1793
query65 3345 3286 3242 3242
query66 1198 330 307 307
query67 15992 15379 15443 15379
query68 10494 535 519 519
query69 585 300 304 300
query70 1752 1560 1568 1560
query71 10415 10201 10203 10201
query72 4829 2811 2819 2811
query73 2133 314 312 312
query74 7062 6405 6435 6405
query75 4491 2368 2331 2331
query76 6280 1065 1039 1039
query77 662 225 237 225
query78 9137 9017 8734 8734
query79 979 506 509 506
query80 508 322 310 310
query81 471 202 200 200
query82 173 81 77 77
query83 135 114 122 114
query84 267 68 68 68
query85 1077 323 322 322
query86 396 405 403 403
query87 3481 3391 3380 3380
query88 2770 2230 2200 2200
query89 416 352 348 348
query90 1949 187 182 182
query91 152 135 132 132
query92 47 42 44 42
query93 936 430 422 422
query94 1177 161 156 156
query95 487 457 453 453
query96 591 330 320 320
query97 4279 4153 4176 4153
query98 194 200 184 184
query99 985 676 698 676
Total cold run time: 294473 ms
Total hot run time: 187428 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org