You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "TangSiyang2001 (via GitHub)" <gi...@apache.org> on 2024/02/24 04:04:46 UTC
[PR] [cherry-pick](stmt-forward) record query result for proxy query to avoid EOF (#30536) [doris]
TangSiyang2001 opened a new pull request, #31363:
URL: https://github.com/apache/doris/pull/31363
## Proposed changes
pick: #30536
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [cherry-pick](stmt-forward) record query result for proxy query to avoid EOF (#30536) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31363:
URL: https://github.com/apache/doris/pull/31363#issuecomment-1962253907
<details>
<summary>TPC-DS: <b>Total hot run time: 202445 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 04da285c18dd937b09664e1bf96b019ecd32eb16, data reload: false
query1 914 398 377 377
query2 6535 2048 2215 2048
query3 6906 203 197 197
query4 20172 18049 18038 18038
query5 19703 6508 6522 6508
query6 301 218 224 218
query7 4358 311 304 304
query8 242 257 229 229
query9 3117 2697 2634 2634
query10 414 299 298 298
query11 11281 10678 10681 10678
query12 127 76 75 75
query13 5577 662 639 639
query14 17894 13532 13773 13532
query15 380 230 248 230
query16 6434 265 264 264
query17 1479 1514 871 871
query18 2267 414 412 412
query19 195 144 144 144
query20 81 75 76 75
query21 187 92 101 92
query22 5248 5063 5122 5063
query23 32691 31822 32054 31822
query24 6840 6521 6458 6458
query25 520 406 412 406
query26 457 166 157 157
query27 1317 299 297 297
query28 6029 2241 2209 2209
query29 2864 2793 2861 2793
query30 236 160 158 158
query31 848 721 723 721
query32 68 58 56 56
query33 331 243 242 242
query34 819 479 473 473
query35 1118 876 895 876
query36 1682 1483 1585 1483
query37 89 60 60 60
query38 3083 2947 2929 2929
query39 1374 1319 1340 1319
query40 207 96 93 93
query41 34 31 35 31
query42 85 79 86 79
query43 652 584 649 584
query44 1081 714 723 714
query45 244 234 226 226
query46 1239 998 980 980
query47 1912 1843 1610 1610
query48 964 688 658 658
query49 613 373 366 366
query50 868 704 618 618
query51 4809 4711 4680 4680
query52 88 80 74 74
query53 450 323 322 322
query54 2656 2440 2457 2440
query55 88 72 87 72
query56 214 203 208 203
query57 1205 1085 1110 1085
query58 209 198 196 196
query59 3747 3469 3285 3285
query60 200 185 215 185
query61 83 84 83 83
query62 811 494 482 482
query63 475 346 337 337
query64 2091 1436 1347 1347
query65 3666 3556 3577 3556
query66 769 383 356 356
query67 16361 15711 15472 15472
query68 9799 647 664 647
query69 564 339 348 339
query70 1951 1700 1630 1630
query71 393 292 307 292
query72 6515 3452 3423 3423
query73 736 317 313 313
query74 6273 5934 5793 5793
query75 5361 3741 3794 3741
query76 6007 1139 1217 1139
query77 1041 249 241 241
query78 12608 12564 11735 11735
query79 11349 668 653 653
query80 693 383 381 381
query81 479 232 237 232
query82 481 99 98 98
query83 178 129 129 129
query84 244 67 68 67
query85 735 278 280 278
query86 398 422 376 376
query87 3269 3051 3004 3004
query88 4922 2346 2337 2337
query89 415 289 275 275
query90 2069 194 217 194
query91 159 122 117 117
query92 54 51 51 51
query93 4851 561 564 561
query94 872 209 206 206
query95 1097 1078 1057 1057
query96 659 332 323 323
query97 6408 6279 6349 6279
query98 187 183 167 167
query99 2942 894 848 848
Total cold run time: 313708 ms
Total hot run time: 202445 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [cherry-pick](stmt-forward) record query result for proxy query to avoid EOF (#30536) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31363:
URL: https://github.com/apache/doris/pull/31363#issuecomment-1962243488
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [cherry-pick](stmt-forward) record query result for proxy query to avoid EOF (#30536) [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31363:
URL: https://github.com/apache/doris/pull/31363#issuecomment-1962245761
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [cherry-pick](stmt-forward) record query result for proxy query to avoid EOF (#30536) [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31363:
URL: https://github.com/apache/doris/pull/31363#issuecomment-1962247747
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [cherry-pick](stmt-forward) record query result for proxy query to avoid EOF (#30536) [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #31363:
URL: https://github.com/apache/doris/pull/31363
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [cherry-pick](stmt-forward) record query result for proxy query to avoid EOF (#30536) [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31363:
URL: https://github.com/apache/doris/pull/31363#issuecomment-1962246594
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [cherry-pick](stmt-forward) record query result for proxy query to avoid EOF (#30536) [doris]
Posted by "TangSiyang2001 (via GitHub)" <gi...@apache.org>.
TangSiyang2001 commented on PR #31363:
URL: https://github.com/apache/doris/pull/31363#issuecomment-1962246501
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [cherry-pick](stmt-forward) record query result for proxy query to avoid EOF (#30536) [doris]
Posted by "TangSiyang2001 (via GitHub)" <gi...@apache.org>.
TangSiyang2001 commented on PR #31363:
URL: https://github.com/apache/doris/pull/31363#issuecomment-1962244462
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [cherry-pick](stmt-forward) record query result for proxy query to avoid EOF (#30536) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31363:
URL: https://github.com/apache/doris/pull/31363#issuecomment-1962251604
<details>
<summary>TPC-H: <b>Total hot run time: 49953 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 04da285c18dd937b09664e1bf96b019ecd32eb16, data reload: false
------ Round 1 ----------------------------------
q1 18378 4499 4353 4353
q2 2036 152 140 140
q3 10465 1915 1964 1915
q4 10331 1239 1306 1239
q5 8598 4360 3987 3987
q6 236 124 124 124
q7 2031 1584 1581 1581
q8 9523 2734 2736 2734
q9 11694 10659 10380 10380
q10 8679 3529 3533 3529
q11 418 243 246 243
q12 467 303 302 302
q13 18354 3961 4022 3961
q14 362 334 312 312
q15 514 455 460 455
q16 696 598 587 587
q17 1137 981 925 925
q18 7335 6951 6852 6852
q19 1704 1605 1503 1503
q20 535 307 299 299
q21 4524 4147 4133 4133
q22 513 419 399 399
Total cold run time: 118530 ms
Total hot run time: 49953 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4339 4311 4327 4311
q2 320 218 219 218
q3 4176 4172 4158 4158
q4 2763 2748 2753 2748
q5 7269 7184 7191 7184
q6 236 124 121 121
q7 3288 2897 2842 2842
q8 4338 4452 4518 4452
q9 17133 16816 16914 16816
q10 4221 4326 4274 4274
q11 738 688 675 675
q12 1023 854 835 835
q13 6707 3726 3734 3726
q14 455 418 437 418
q15 493 461 457 457
q16 766 715 703 703
q17 3782 3932 3933 3932
q18 8816 8732 8785 8732
q19 1738 1702 1669 1669
q20 2410 2125 2108 2108
q21 8518 8672 8519 8519
q22 1084 986 980 980
Total cold run time: 84613 ms
Total hot run time: 79878 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [cherry-pick](stmt-forward) record query result for proxy query to avoid EOF (#30536) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31363:
URL: https://github.com/apache/doris/pull/31363#issuecomment-1962254884
<details>
<summary>ClickBench: <b>Total hot run time: 30.44 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 04da285c18dd937b09664e1bf96b019ecd32eb16, data reload: false
query1 0.03 0.02 0.02
query2 0.06 0.02 0.01
query3 0.25 0.04 0.05
query4 1.81 0.07 0.06
query5 0.53 0.52 0.52
query6 1.26 0.62 0.61
query7 0.02 0.01 0.02
query8 0.03 0.02 0.02
query9 0.53 0.50 0.49
query10 0.55 0.53 0.54
query11 0.12 0.09 0.09
query12 0.12 0.10 0.10
query13 0.62 0.62 0.62
query14 0.79 0.78 0.79
query15 0.78 0.75 0.76
query16 0.37 0.36 0.37
query17 1.01 1.01 0.99
query18 0.26 0.23 0.26
query19 1.94 1.78 1.89
query20 0.02 0.00 0.01
query21 15.45 0.55 0.58
query22 2.29 1.97 1.33
query23 17.34 0.93 1.00
query24 7.77 1.33 0.94
query25 1.53 0.13 0.10
query26 0.41 0.15 0.15
query27 0.12 0.11 0.10
query28 6.37 0.72 0.72
query29 12.66 2.28 2.33
query30 0.57 0.50 0.51
query31 2.82 0.37 0.39
query32 3.41 0.50 0.51
query33 3.09 3.07 3.10
query34 15.24 4.78 4.81
query35 4.88 4.85 4.87
query36 1.06 1.01 1.02
query37 0.05 0.05 0.04
query38 0.04 0.02 0.02
query39 0.01 0.01 0.01
query40 0.16 0.14 0.14
query41 0.06 0.02 0.02
query42 0.02 0.01 0.01
query43 0.02 0.02 0.01
Total cold run time: 106.47 s
Total hot run time: 30.44 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [cherry-pick](stmt-forward) record query result for proxy query to avoid EOF (#30536) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31363:
URL: https://github.com/apache/doris/pull/31363#issuecomment-1962255600
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 04da285c18dd937b09664e1bf96b019ecd32eb16 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 21.5 seconds inserted 10000000 Rows, about 465K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [cherry-pick](stmt-forward) record query result for proxy query to avoid EOF (#30536) [doris]
Posted by "TangSiyang2001 (via GitHub)" <gi...@apache.org>.
TangSiyang2001 commented on PR #31363:
URL: https://github.com/apache/doris/pull/31363#issuecomment-1962300401
run feut
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org