You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Mryange (via GitHub)" <gi...@apache.org> on 2024/03/30 03:56:19 UTC
[PR] [log](pipeline)add more log in scan localstate [doris]
Mryange opened a new pull request, #33062:
URL: https://github.com/apache/doris/pull/33062
## Proposed changes
error log
```
PipelineTask[this = 0x61901570a780, state = BLOCKED_FOR_SOURCE, dry run = false, elapse time = 599.986904548s], block dependency = OLAP_SCAN_OPERATOR_DEPENDENCY: id=4, block task = 1, ready=false, _always_ready=false, is cancelled=false, is running = false
operators:
OLAP_SCAN_OPERATOR: id=4, parallel_tasks=1, _eos = false
DATA_STREAM_SINK_OPERATOR: id=5, Sink Buffer: (_should_stop = false, _busy_channels = 0)
Read Dependency Information:
```
parallel_tasks=1, _eos = false with out _scanner_ctx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [log](pipeline)add more log in scan localstate [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #33062:
URL: https://github.com/apache/doris/pull/33062#issuecomment-2027910599
<details>
<summary>TPC-DS: <b>Total hot run time: 182420 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9167b2c2dcafbf8c06e630270fd4cdcec651454a, data reload: false
query1 897 1116 1132 1116
query2 7201 1898 1881 1881
query3 6653 207 206 206
query4 33118 21578 21333 21333
query5 4256 403 415 403
query6 223 182 168 168
query7 3976 293 284 284
query8 221 171 177 171
query9 5537 2315 2289 2289
query10 484 232 252 232
query11 14799 14268 14256 14256
query12 148 92 88 88
query13 970 368 372 368
query14 8758 6822 6892 6822
query15 227 173 193 173
query16 7571 263 258 258
query17 1581 576 564 564
query18 1906 290 280 280
query19 214 161 154 154
query20 95 86 90 86
query21 206 127 130 127
query22 5022 4880 4825 4825
query23 33355 33028 32874 32874
query24 12787 2803 2856 2803
query25 655 384 389 384
query26 1890 151 154 151
query27 3290 319 321 319
query28 7699 1834 1817 1817
query29 993 629 624 624
query30 284 151 149 149
query31 908 713 696 696
query32 63 55 53 53
query33 588 238 238 238
query34 947 466 468 466
query35 839 676 684 676
query36 1049 916 898 898
query37 262 66 70 66
query38 3573 3486 3433 3433
query39 1572 1556 1534 1534
query40 278 126 124 124
query41 47 46 43 43
query42 106 94 96 94
query43 490 451 447 447
query44 1105 720 699 699
query45 269 248 262 248
query46 1076 676 673 673
query47 1936 1879 1897 1879
query48 381 305 287 287
query49 1159 364 363 363
query50 768 379 370 370
query51 6767 6612 6658 6612
query52 108 91 95 91
query53 342 275 277 275
query54 272 230 243 230
query55 85 80 78 78
query56 241 216 222 216
query57 1221 1130 1142 1130
query58 228 207 204 204
query59 2835 2686 2641 2641
query60 257 237 241 237
query61 103 97 96 96
query62 635 448 451 448
query63 310 280 281 280
query64 6436 4154 4156 4154
query65 3122 3120 3041 3041
query66 1376 368 371 368
query67 15456 15299 15305 15299
query68 6428 531 517 517
query69 552 314 316 314
query70 1204 1131 1220 1131
query71 483 279 277 277
query72 6746 2771 2589 2589
query73 778 319 332 319
query74 6935 6476 6388 6388
query75 3466 2335 2382 2335
query76 4103 937 993 937
query77 678 265 256 256
query78 10897 10221 10180 10180
query79 8574 533 505 505
query80 2119 413 414 413
query81 529 218 211 211
query82 1427 94 90 90
query83 335 167 164 164
query84 256 83 83 83
query85 1675 264 258 258
query86 505 300 315 300
query87 3822 3543 3526 3526
query88 5404 2301 2299 2299
query89 538 358 362 358
query90 1950 179 178 178
query91 128 96 96 96
query92 63 46 48 46
query93 6864 512 498 498
query94 1300 182 178 178
query95 396 304 300 300
query96 589 266 266 266
query97 2644 2454 2471 2454
query98 249 228 212 212
query99 1298 850 854 850
Total cold run time: 305161 ms
Total hot run time: 182420 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [log](pipeline)add more log in scan localstate [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #33062:
URL: https://github.com/apache/doris/pull/33062#issuecomment-2027904623
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [log](pipeline)add more log in scan localstate [doris]
Posted by "BiteTheDDDDt (via GitHub)" <gi...@apache.org>.
BiteTheDDDDt merged PR #33062:
URL: https://github.com/apache/doris/pull/33062
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [log](pipeline)add more log in scan localstate [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #33062:
URL: https://github.com/apache/doris/pull/33062#issuecomment-2027911894
<details>
<summary>ClickBench: <b>Total hot run time: 30.3 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9167b2c2dcafbf8c06e630270fd4cdcec651454a, data reload: false
query1 0.05 0.04 0.03
query2 0.08 0.04 0.04
query3 0.22 0.05 0.05
query4 1.68 0.06 0.07
query5 0.50 0.49 0.50
query6 1.15 0.68 0.65
query7 0.02 0.01 0.02
query8 0.06 0.04 0.04
query9 0.55 0.49 0.49
query10 0.54 0.54 0.54
query11 0.16 0.10 0.11
query12 0.13 0.11 0.11
query13 0.59 0.58 0.58
query14 0.76 0.78 0.76
query15 0.83 0.80 0.81
query16 0.38 0.36 0.38
query17 0.97 0.97 0.97
query18 0.23 0.22 0.25
query19 1.87 1.75 1.66
query20 0.02 0.01 0.01
query21 15.56 0.64 0.64
query22 2.65 2.69 2.59
query23 17.24 0.98 0.80
query24 1.16 0.26 0.21
query25 0.10 0.08 0.08
query26 0.22 0.16 0.17
query27 0.08 0.08 0.07
query28 13.96 0.94 0.92
query29 12.54 3.20 3.19
query30 0.27 0.09 0.10
query31 2.80 0.38 0.38
query32 3.29 0.46 0.45
query33 2.82 2.81 2.81
query34 16.69 4.43 4.40
query35 4.48 4.49 4.52
query36 0.61 0.46 0.47
query37 0.18 0.15 0.15
query38 0.15 0.14 0.14
query39 0.06 0.03 0.04
query40 0.19 0.16 0.15
query41 0.08 0.05 0.05
query42 0.05 0.04 0.05
query43 0.05 0.04 0.03
Total cold run time: 106.02 s
Total hot run time: 30.3 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [log](pipeline)add more log in scan localstate [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #33062:
URL: https://github.com/apache/doris/pull/33062#issuecomment-2042555821
TeamCity be ut coverage result:
Function Coverage: 35.58% (8887/24976)
Line Coverage: 27.33% (72975/267018)
Region Coverage: 26.50% (37718/142350)
Branch Coverage: 23.31% (19220/82468)
Coverage Report: http://coverage.selectdb-in.cc/coverage/9167b2c2dcafbf8c06e630270fd4cdcec651454a_9167b2c2dcafbf8c06e630270fd4cdcec651454a/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [log](pipeline)add more log in scan localstate [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #33062:
URL: https://github.com/apache/doris/pull/33062#issuecomment-2027903991
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [log](pipeline)add more log in scan localstate [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #33062:
URL: https://github.com/apache/doris/pull/33062#issuecomment-2042399659
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [log](pipeline)add more log in scan localstate [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #33062:
URL: https://github.com/apache/doris/pull/33062#issuecomment-2027902501
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [log](pipeline)add more log in scan localstate [doris]
Posted by "Mryange (via GitHub)" <gi...@apache.org>.
Mryange commented on PR #33062:
URL: https://github.com/apache/doris/pull/33062#issuecomment-2042398187
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [log](pipeline)add more log in scan localstate [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #33062:
URL: https://github.com/apache/doris/pull/33062#issuecomment-2027912906
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 9167b2c2dcafbf8c06e630270fd4cdcec651454a with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.6 seconds inserted 10000000 Rows, about 735K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [log](pipeline)add more log in scan localstate [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #33062:
URL: https://github.com/apache/doris/pull/33062#issuecomment-2042399594
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [log](pipeline)add more log in scan localstate [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #33062:
URL: https://github.com/apache/doris/pull/33062#issuecomment-2027908216
<details>
<summary>TPC-H: <b>Total hot run time: 38555 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9167b2c2dcafbf8c06e630270fd4cdcec651454a, data reload: false
------ Round 1 ----------------------------------
q1 17970 9316 4401 4401
q2 2584 201 192 192
q3 11072 1153 1153 1153
q4 10988 844 676 676
q5 7550 2863 2815 2815
q6 227 136 135 135
q7 1050 651 613 613
q8 9561 2076 2079 2076
q9 7618 6793 6609 6609
q10 8514 3504 3511 3504
q11 481 241 234 234
q12 473 213 214 213
q13 17793 2915 2956 2915
q14 276 234 241 234
q15 512 481 474 474
q16 536 386 390 386
q17 982 539 602 539
q18 7234 6711 6684 6684
q19 5108 1424 1448 1424
q20 662 307 308 307
q21 3523 2671 2752 2671
q22 362 300 311 300
Total cold run time: 115076 ms
Total hot run time: 38555 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4367 4228 4249 4228
q2 388 278 267 267
q3 3001 2736 2725 2725
q4 1834 1498 1552 1498
q5 5307 5249 5279 5249
q6 210 122 124 122
q7 2251 1881 1885 1881
q8 3223 3292 3252 3252
q9 8633 8508 8540 8508
q10 3899 3724 3734 3724
q11 568 484 472 472
q12 747 600 630 600
q13 17442 2939 2982 2939
q14 302 278 257 257
q15 518 462 470 462
q16 494 433 427 427
q17 1790 1436 1413 1413
q18 7676 7402 7395 7395
q19 1626 1533 1457 1457
q20 1939 1744 1741 1741
q21 5031 4893 4890 4890
q22 539 454 429 429
Total cold run time: 71785 ms
Total hot run time: 53936 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [log](pipeline)add more log in scan localstate [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #33062:
URL: https://github.com/apache/doris/pull/33062#issuecomment-2027909349
TeamCity be ut coverage result:
Function Coverage: 35.55% (8840/24864)
Line Coverage: 27.29% (72481/265608)
Region Coverage: 26.50% (37508/141564)
Branch Coverage: 23.31% (19127/82058)
Coverage Report: http://coverage.selectdb-in.cc/coverage/9167b2c2dcafbf8c06e630270fd4cdcec651454a_9167b2c2dcafbf8c06e630270fd4cdcec651454a/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org