You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "yiguolei (via GitHub)" <gi...@apache.org> on 2024/01/09 04:51:47 UTC
[PR] [debug](timeout) add more log in scanner ctx to find timeout problem [doris]
yiguolei opened a new pull request, #29704:
URL: https://github.com/apache/doris/pull/29704
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](timeout) add more log in scanner ctx to find timeout problem [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #29704:
URL: https://github.com/apache/doris/pull/29704#issuecomment-1882458518
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](timeout) add more log in scanner ctx to find timeout problem [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29704:
URL: https://github.com/apache/doris/pull/29704#issuecomment-1882504581
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.41 seconds
stream load tsv: 567 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.4 seconds inserted 10000000 Rows, about 364K ops/s
storage size: 17183585684 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](timeout) add more log in scanner ctx to find timeout problem [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29704:
URL: https://github.com/apache/doris/pull/29704#issuecomment-1882415572
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](timeout) add more log in scanner ctx to find timeout problem [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #29704:
URL: https://github.com/apache/doris/pull/29704
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](timeout) add more log in scanner ctx to find timeout problem [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29704:
URL: https://github.com/apache/doris/pull/29704#issuecomment-1882541082
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](timeout) add more log in scanner ctx to find timeout problem [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29704:
URL: https://github.com/apache/doris/pull/29704#issuecomment-1882492051
<details>
<summary>TPC-H: <b>Total hot run time: 39562 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 680e424f260d6262b7b3fc83d5a5c1a1b702f3c8, data reload: false
------ Round 1 ----------------------------------
q1 18328 5560 5520 5520
q2 2059 166 156 156
q3 10616 1116 1193 1116
q4 10206 791 799 791
q5 7827 2995 2934 2934
q6 206 134 136 134
q7 905 532 500 500
q8 9283 2010 1984 1984
q9 6521 6400 6397 6397
q10 8238 3067 3093 3067
q11 435 235 223 223
q12 392 233 241 233
q13 18284 3494 3469 3469
q14 236 213 212 212
q15 554 517 505 505
q16 476 409 393 393
q17 937 748 673 673
q18 7300 6728 6697 6697
q19 1547 1519 1506 1506
q20 741 316 314 314
q21 2771 2444 2403 2403
q22 387 338 335 335
Total cold run time: 108249 ms
Total hot run time: 39562 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5566 5496 5501 5496
q2 359 270 255 255
q3 3343 3285 3345 3285
q4 2118 2136 2102 2102
q5 5848 5836 5841 5836
q6 210 122 126 122
q7 2327 1902 1855 1855
q8 3395 3551 3525 3525
q9 8956 8985 8944 8944
q10 3844 3896 3852 3852
q11 597 522 514 514
q12 819 649 648 648
q13 6840 3278 3241 3241
q14 302 258 253 253
q15 544 529 514 514
q16 558 488 503 488
q17 2397 2398 2390 2390
q18 8753 8722 8622 8622
q19 1578 1605 1602 1602
q20 2215 1967 1962 1962
q21 5672 5483 5290 5290
q22 566 519 508 508
Total cold run time: 66807 ms
Total hot run time: 61304 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](timeout) add more log in scanner ctx to find timeout problem [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29704:
URL: https://github.com/apache/doris/pull/29704#issuecomment-1882484483
TeamCity be ut coverage result:
Function Coverage: 36.66% (8639/23567)
Line Coverage: 28.68% (70232/244895)
Region Coverage: 27.63% (36324/131470)
Branch Coverage: 24.32% (18562/76330)
Coverage Report: http://coverage.selectdb-in.cc/coverage/680e424f260d6262b7b3fc83d5a5c1a1b702f3c8_680e424f260d6262b7b3fc83d5a5c1a1b702f3c8/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](timeout) add more log in scanner ctx to find timeout problem [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29704:
URL: https://github.com/apache/doris/pull/29704#issuecomment-1882502240
<details>
<summary>TPC-DS: <b>Total hot run time: 187040 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 680e424f260d6262b7b3fc83d5a5c1a1b702f3c8, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 970 346 340 340
query2 6869 2154 1997 1997
query3 6726 230 228 228
query4 25893 22624 22531 22531
query5 6112 615 618 615
query6 337 207 221 207
query7 4635 306 318 306
query8 269 240 240 240
query9 9652 3349 3318 3318
query10 595 349 345 345
query11 16346 15740 15648 15648
query12 183 118 120 118
query13 1674 341 360 341
query14 12738 7595 7584 7584
query15 257 240 262 240
query16 6356 363 323 323
query17 1712 550 538 538
query18 1980 322 335 322
query19 240 172 171 171
query20 135 121 130 121
query21 189 95 103 95
query22 4925 4584 4572 4572
query23 32150 31459 31483 31459
query24 12780 3057 3003 3003
query25 616 399 391 391
query26 1853 187 186 186
query27 2919 315 318 315
query28 7017 2175 2118 2118
query29 2112 452 434 434
query30 288 146 148 146
query31 1070 873 855 855
query32 141 102 104 102
query33 847 360 364 360
query34 872 495 499 495
query35 1040 965 886 886
query36 1382 1331 1323 1323
query37 228 116 108 108
query38 3464 3392 3333 3333
query39 1397 1294 1302 1294
query40 316 100 96 96
query41 41 40 38 38
query42 121 125 119 119
query43 618 611 603 603
query44 1307 833 843 833
query45 220 215 209 209
query46 1076 737 708 708
query47 1755 1768 1752 1752
query48 394 288 298 288
query49 1216 347 334 334
query50 732 350 375 350
query51 5573 5481 5343 5343
query52 132 108 111 108
query53 248 194 191 191
query54 1380 749 733 733
query55 122 112 103 103
query56 284 280 268 268
query57 1057 1062 1054 1054
query58 354 330 356 330
query59 3272 3069 3132 3069
query60 349 330 327 327
query61 131 157 148 148
query62 617 495 478 478
query63 220 190 207 190
query64 6120 1776 1755 1755
query65 3461 3389 3422 3389
query66 1336 468 453 453
query67 15710 15769 15174 15174
query68 12568 543 562 543
query69 636 355 362 355
query70 1908 1756 1751 1751
query71 634 306 308 306
query72 5446 3482 3504 3482
query73 3035 348 344 344
query74 7188 6643 6724 6643
query75 5304 2314 2261 2261
query76 6464 1130 1224 1130
query77 754 357 335 335
query78 9060 8869 8549 8549
query79 1395 553 559 553
query80 560 355 360 355
query81 460 218 218 218
query82 242 128 146 128
query83 194 151 141 141
query84 265 62 58 58
query85 937 290 274 274
query86 478 449 454 449
query87 3657 3530 3468 3468
query88 3565 2535 2539 2535
query89 405 311 318 311
query90 1923 277 253 253
query91 121 95 96 95
query92 115 91 99 91
query93 1327 483 487 483
query94 903 242 256 242
query95 580 531 509 509
query96 645 349 358 349
query97 4417 4251 4351 4251
query98 243 234 222 222
query99 1156 905 905 905
Total cold run time: 303716 ms
Total hot run time: 187040 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [debug](timeout) add more log in scanner ctx to find timeout problem [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29704:
URL: https://github.com/apache/doris/pull/29704#issuecomment-1882541048
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org