You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "TangSiyang2001 (via GitHub)" <gi...@apache.org> on 2024/01/29 15:22:12 UTC
[PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
TangSiyang2001 opened a new pull request, #30536:
URL: https://github.com/apache/doris/pull/30536
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1915020924
<details>
<summary>TPC-H: <b>Total hot run time: 38354 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit eb1dcc424240763422b59b495ddd719b15d44096, data reload: false
------ Round 1 ----------------------------------
q1 17708 5532 5214 5214
q2 2094 159 140 140
q3 11120 958 955 955
q4 4844 794 832 794
q5 7906 2994 3022 2994
q6 193 129 122 122
q7 1178 762 741 741
q8 9310 2117 2086 2086
q9 7611 6403 6346 6346
q10 8129 2452 2411 2411
q11 420 196 213 196
q12 735 283 284 283
q13 18037 3339 3321 3321
q14 268 258 240 240
q15 530 481 492 481
q16 453 415 418 415
q17 953 531 514 514
q18 7051 5992 6331 5992
q19 1585 1405 1385 1385
q20 600 347 321 321
q21 7028 3134 3098 3098
q22 818 318 305 305
Total cold run time: 108571 ms
Total hot run time: 38354 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5110 5020 5135 5020
q2 349 233 224 224
q3 2966 2806 2677 2677
q4 1807 1513 1587 1513
q5 5241 5293 5303 5293
q6 197 116 116 116
q7 2113 1750 1807 1750
q8 3124 3196 3254 3196
q9 8668 8666 12327 8666
q10 5426 3480 3521 3480
q11 549 451 476 451
q12 754 586 597 586
q13 16967 3125 3071 3071
q14 279 263 273 263
q15 531 508 497 497
q16 501 461 468 461
q17 1891 1818 1646 1646
q18 9002 15680 8748 8748
q19 27403 1566 1555 1555
q20 4603 1902 1888 1888
q21 15560 4841 4928 4841
q22 1281 483 498 483
Total cold run time: 114322 ms
Total hot run time: 56425 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1916501162
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1914940730
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1929023418
<details>
<summary>ClickBench: <b>Total hot run time: 29.92 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6fcf412f939e1dae00ac440b45ee6a5df9f40f3e, data reload: false
query1 0.04 0.03 0.03
query2 0.06 0.02 0.02
query3 0.22 0.06 0.06
query4 1.68 0.09 0.10
query5 0.54 0.52 0.52
query6 1.19 0.65 0.63
query7 0.02 0.01 0.01
query8 0.03 0.02 0.02
query9 0.54 0.50 0.50
query10 0.56 0.56 0.54
query11 0.12 0.08 0.08
query12 0.11 0.09 0.09
query13 0.60 0.61 0.61
query14 0.80 0.78 0.79
query15 0.79 0.77 0.77
query16 0.39 0.38 0.40
query17 1.05 1.00 0.99
query18 0.23 0.26 0.23
query19 1.86 1.80 1.77
query20 0.02 0.01 0.01
query21 15.39 0.59 0.55
query22 2.72 2.51 1.61
query23 17.19 0.78 0.89
query24 2.46 1.02 0.26
query25 0.40 0.10 0.08
query26 0.41 0.14 0.14
query27 0.09 0.04 0.06
query28 12.30 0.85 0.83
query29 12.57 3.06 3.06
query30 0.64 0.52 0.55
query31 2.81 0.34 0.34
query32 3.37 0.47 0.48
query33 3.22 3.22 3.24
query34 15.97 4.31 4.28
query35 4.37 4.29 4.30
query36 1.11 1.06 1.05
query37 0.06 0.05 0.06
query38 0.03 0.02 0.02
query39 0.02 0.02 0.02
query40 0.17 0.13 0.13
query41 0.07 0.02 0.02
query42 0.02 0.02 0.01
query43 0.03 0.02 0.02
Total cold run time: 106.27 s
Total hot run time: 29.92 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1915047662
<details>
<summary>TPC-DS: <b>Total hot run time: 173211 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit eb1dcc424240763422b59b495ddd719b15d44096, data reload: false
query1 933 333 326 326
query2 6567 1950 1805 1805
query3 6707 204 199 199
query4 33088 22100 21963 21963
query5 4445 361 413 361
query6 249 152 150 150
query7 4607 259 261 259
query8 234 185 178 178
query9 8537 2290 2277 2277
query10 407 197 199 197
query11 17802 15397 15276 15276
query12 117 66 63 63
query13 1696 381 362 362
query14 9194 6498 6495 6495
query15 211 177 177 177
query16 5690 257 247 247
query17 954 481 477 477
query18 1798 255 244 244
query19 167 134 130 130
query20 76 74 72 72
query21 200 137 132 132
query22 5043 4821 4685 4685
query23 31131 30500 30353 30353
query24 12340 2812 2784 2784
query25 587 314 306 306
query26 1974 136 148 136
query27 3179 286 283 283
query28 7793 1819 1812 1812
query29 2207 611 607 607
query30 290 137 138 137
query31 899 715 725 715
query32 81 56 49 49
query33 713 212 211 211
query34 1183 464 470 464
query35 841 754 743 743
query36 1330 1299 1277 1277
query37 91 56 61 56
query38 3293 3210 3200 3200
query39 1328 1240 1240 1240
query40 340 88 84 84
query41 38 37 35 35
query42 81 81 82 81
query43 528 472 476 472
query44 1109 701 698 698
query45 199 180 175 175
query46 1074 656 644 644
query47 1597 1563 1515 1515
query48 388 321 308 308
query49 1224 294 291 291
query50 689 308 309 308
query51 5254 5173 5167 5167
query52 89 83 77 77
query53 314 262 263 262
query54 253 184 183 183
query55 84 75 79 75
query56 185 154 172 154
query57 981 909 884 884
query58 181 154 156 154
query59 2650 2562 2498 2498
query60 199 182 183 182
query61 89 81 81 81
query62 615 365 350 350
query63 272 262 266 262
query64 6074 1746 1750 1746
query65 3292 3223 3215 3215
query66 1481 318 298 298
query67 14612 14413 14157 14157
query68 3709 485 498 485
query69 416 310 294 294
query70 1614 1515 1520 1515
query71 287 214 216 214
query72 3704 2824 2855 2824
query73 679 312 310 310
query74 6602 6274 6280 6274
query75 2738 2306 2290 2290
query76 2564 1035 1007 1007
query77 424 227 227 227
query78 9004 9026 8569 8569
query79 2107 491 490 490
query80 1387 325 314 314
query81 519 195 198 195
query82 679 83 84 83
query83 223 114 116 114
query84 276 74 75 74
query85 2231 338 315 315
query86 514 395 411 395
query87 3489 3310 3244 3244
query88 3782 2169 2164 2164
query89 396 361 360 360
query90 2020 190 184 184
query91 145 118 122 118
query92 56 43 45 43
query93 2101 446 415 415
query94 1445 171 160 160
query95 512 461 454 454
query96 623 317 320 317
query97 4238 4135 4105 4105
query98 203 186 190 186
query99 1053 673 668 668
Total cold run time: 281614 ms
Total hot run time: 173211 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1929028464
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 6fcf412f939e1dae00ac440b45ee6a5df9f40f3e with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 14.4 seconds inserted 10000000 Rows, about 694K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou commented on code in PR #30536:
URL: https://github.com/apache/doris/pull/30536#discussion_r1471552188
##########
fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java:
##########
@@ -2869,6 +2877,24 @@ public String getOriginStmtInString() {
}
return "";
}
+
+ public List<ByteBuffer> getQueryResultBufList() {
+ return ((ProxyMysqlChannel) context.getMysqlChannel()).getProxyResultBufferList();
+ }
+
+ public boolean sendProxyQueryResult() throws IOException {
+ if (masterOpExecutor == null) {
+ return false;
+ }
+ List<ByteBuffer> queryResultBufList = masterOpExecutor.getQueryResultBufList();
+ if (queryResultBufList.isEmpty()) {
+ return false;
+ }
+ for (ByteBuffer byteBuffer : queryResultBufList) {
+ context.getMysqlChannel().sendOnePacket(byteBuffer);
+ }
+ return true;
Review Comment:
```suggestion
for (ByteBuffer byteBuffer : queryResultBufList) {
context.getMysqlChannel().sendOnePacket(byteBuffer);
}
return !queryResultBufList.isEmpty());
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1923893962
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1916590554
<details>
<summary>TPC-H: <b>Total hot run time: 36886 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7387e2d101306fde0ae38f973fe1a21485a9b8da, data reload: false
------ Round 1 ----------------------------------
q1 17632 4695 4429 4429
q2 2070 141 128 128
q3 10671 945 908 908
q4 4676 747 692 692
q5 7681 2806 2805 2805
q6 186 122 124 122
q7 1134 720 720 720
q8 9379 1956 2014 1956
q9 7185 6333 6369 6333
q10 8100 2450 2421 2421
q11 427 208 211 208
q12 799 275 279 275
q13 18014 3331 3374 3331
q14 282 247 263 247
q15 529 491 497 491
q16 465 424 411 411
q17 947 538 507 507
q18 6840 6044 5855 5855
q19 1582 1456 1358 1358
q20 619 322 332 322
q21 6775 3086 3118 3086
q22 803 313 281 281
Total cold run time: 106796 ms
Total hot run time: 36886 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4443 4404 4433 4404
q2 323 238 236 236
q3 2977 2840 2808 2808
q4 1899 1757 1641 1641
q5 5225 5271 5432 5271
q6 193 117 118 117
q7 2165 1800 1758 1758
q8 3146 3267 3288 3267
q9 8336 8309 8302 8302
q10 5933 3547 3584 3547
q11 531 464 468 464
q12 752 576 600 576
q13 13186 3117 3070 3070
q14 279 251 258 251
q15 527 491 495 491
q16 516 483 487 483
q17 1818 1708 1715 1708
q18 7973 7917 7471 7471
q19 9264 1509 1523 1509
q20 2120 1904 1906 1904
q21 4904 4641 4655 4641
q22 568 456 476 456
Total cold run time: 77078 ms
Total hot run time: 54375 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1926631893
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1928929410
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "TangSiyang2001 (via GitHub)" <gi...@apache.org>.
TangSiyang2001 commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1916480756
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1926631832
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1929014969
<details>
<summary>TPC-DS: <b>Total hot run time: 174635 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6fcf412f939e1dae00ac440b45ee6a5df9f40f3e, data reload: false
query1 940 353 341 341
query2 6523 1958 1900 1900
query3 6699 211 207 207
query4 31653 21922 21963 21922
query5 4314 425 419 419
query6 252 164 166 164
query7 4602 288 284 284
query8 242 174 175 174
query9 8975 2267 2247 2247
query10 411 214 203 203
query11 18437 15392 15253 15253
query12 131 80 76 76
query13 1634 413 409 409
query14 9269 6729 6715 6715
query15 232 176 185 176
query16 8093 257 249 249
query17 1853 545 502 502
query18 2112 268 260 260
query19 362 148 139 139
query20 81 78 79 78
query21 194 136 133 133
query22 5102 4961 4930 4930
query23 31038 29981 29990 29981
query24 8341 2774 2760 2760
query25 538 346 356 346
query26 709 150 147 147
query27 2172 298 303 298
query28 5498 1848 1839 1839
query29 845 591 587 587
query30 279 130 139 130
query31 934 715 705 705
query32 95 54 48 48
query33 531 221 213 213
query34 817 451 459 451
query35 863 779 744 744
query36 1038 903 925 903
query37 100 53 57 53
query38 3226 3152 3119 3119
query39 1311 1254 1243 1243
query40 186 92 93 92
query41 39 35 34 34
query42 99 88 88 88
query43 527 471 466 466
query44 1005 683 694 683
query45 186 182 174 174
query46 1061 657 660 657
query47 1619 1535 1536 1535
query48 431 356 375 356
query49 1058 285 281 281
query50 752 388 391 388
query51 5267 5129 5248 5129
query52 106 80 86 80
query53 329 263 266 263
query54 262 211 215 211
query55 80 73 77 73
query56 220 194 187 187
query57 998 928 908 908
query58 209 182 173 173
query59 2550 2488 2470 2470
query60 236 209 208 208
query61 85 87 81 81
query62 612 362 340 340
query63 288 272 270 270
query64 4740 3752 3577 3577
query65 3264 3233 3221 3221
query66 822 315 303 303
query67 14770 14608 14106 14106
query68 4111 545 535 535
query69 462 323 323 323
query70 1295 1174 1184 1174
query71 315 239 245 239
query72 5990 2845 2674 2674
query73 702 324 321 321
query74 6605 6318 6272 6272
query75 3024 2358 2372 2358
query76 2558 970 952 952
query77 343 236 222 222
query78 9210 8871 8529 8529
query79 3051 494 516 494
query80 2105 349 336 336
query81 533 196 199 196
query82 828 86 83 83
query83 261 119 121 119
query84 283 82 84 82
query85 1791 325 327 325
query86 484 303 320 303
query87 3395 3229 3158 3158
query88 4173 2312 2323 2312
query89 443 351 375 351
query90 1931 162 163 162
query91 151 128 121 121
query92 58 44 42 42
query93 4232 494 468 468
query94 1180 175 172 172
query95 8214 7880 356 356
query96 605 272 267 267
query97 4214 4100 4139 4100
query98 207 205 189 189
query99 1190 698 699 698
Total cold run time: 286191 ms
Total hot run time: 174635 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1916609277
<details>
<summary>TPC-DS: <b>Total hot run time: 172917 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7387e2d101306fde0ae38f973fe1a21485a9b8da, data reload: false
query1 936 340 330 330
query2 6564 1876 1876 1876
query3 6700 208 202 202
query4 27181 22140 22064 22064
query5 4448 430 412 412
query6 248 158 156 156
query7 4604 270 253 253
query8 242 183 200 183
query9 9099 2257 2237 2237
query10 406 206 194 194
query11 17297 15303 15246 15246
query12 122 68 68 68
query13 1671 372 366 366
query14 9325 6616 6526 6526
query15 216 185 190 185
query16 6430 260 250 250
query17 1899 472 475 472
query18 1927 254 246 246
query19 181 132 131 131
query20 76 69 66 66
query21 199 133 120 120
query22 4665 4641 4732 4641
query23 31422 30573 30388 30388
query24 8627 2729 2739 2729
query25 531 305 320 305
query26 830 142 134 134
query27 2831 291 281 281
query28 5996 1853 1838 1838
query29 963 628 623 623
query30 285 134 146 134
query31 916 696 719 696
query32 92 48 48 48
query33 495 210 209 209
query34 835 458 457 457
query35 845 730 757 730
query36 1220 1193 1135 1135
query37 94 56 57 56
query38 3306 3163 3181 3163
query39 1302 1257 1277 1257
query40 194 82 84 82
query41 37 36 34 34
query42 89 80 89 80
query43 517 463 490 463
query44 1017 674 673 673
query45 192 182 175 175
query46 1054 636 652 636
query47 1605 1514 1504 1504
query48 388 306 306 306
query49 1111 291 270 270
query50 682 302 307 302
query51 5312 5178 5196 5178
query52 85 80 78 78
query53 334 258 258 258
query54 226 184 184 184
query55 83 75 76 75
query56 183 159 157 157
query57 984 921 872 872
query58 184 155 156 155
query59 2399 2364 2313 2313
query60 202 176 182 176
query61 93 83 83 83
query62 618 350 347 347
query63 284 264 270 264
query64 4093 1780 1765 1765
query65 3302 3240 3211 3211
query66 1134 320 342 320
query67 14421 14245 14144 14144
query68 4287 516 490 490
query69 437 309 298 298
query70 1572 1470 1519 1470
query71 275 217 224 217
query72 3839 2818 2856 2818
query73 685 319 312 312
query74 6630 6287 6176 6176
query75 2832 2267 2329 2267
query76 3036 1002 907 907
query77 374 233 229 229
query78 8982 8817 8726 8726
query79 2700 491 500 491
query80 1334 328 319 319
query81 534 194 199 194
query82 1024 82 83 82
query83 219 120 119 119
query84 235 70 76 70
query85 1155 327 333 327
query86 462 378 395 378
query87 3443 3299 3264 3264
query88 3766 2272 2263 2263
query89 421 350 342 342
query90 2006 188 190 188
query91 149 119 115 115
query92 52 44 43 43
query93 1918 450 409 409
query94 1264 158 158 158
query95 500 463 460 460
query96 622 326 325 325
query97 4245 4069 4141 4069
query98 209 198 193 193
query99 1021 693 686 686
Total cold run time: 266077 ms
Total hot run time: 172917 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1916618062
<details>
<summary>ClickBench: <b>Total hot run time: 31.09 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7387e2d101306fde0ae38f973fe1a21485a9b8da, data reload: false
query1 0.04 0.03 0.02
query2 0.07 0.03 0.02
query3 0.24 0.06 0.06
query4 1.64 0.09 0.09
query5 0.53 0.52 0.53
query6 1.21 0.65 0.63
query7 0.02 0.01 0.01
query8 0.04 0.03 0.03
query9 0.56 0.48 0.50
query10 0.55 0.56 0.55
query11 0.12 0.08 0.09
query12 0.11 0.09 0.10
query13 0.60 0.60 0.62
query14 0.78 0.79 0.82
query15 0.82 0.78 0.78
query16 0.39 0.38 0.39
query17 1.00 0.99 1.03
query18 0.23 0.28 0.21
query19 1.93 1.77 1.77
query20 0.02 0.01 0.01
query21 15.40 0.59 0.58
query22 2.67 2.52 2.31
query23 17.00 0.81 0.99
query24 2.49 1.03 0.45
query25 0.27 0.21 0.18
query26 0.40 0.15 0.13
query27 0.08 0.05 0.05
query28 12.04 0.86 0.85
query29 12.48 3.16 3.20
query30 0.67 0.57 0.58
query31 2.78 0.34 0.37
query32 3.34 0.48 0.48
query33 3.18 3.22 3.20
query34 15.69 4.43 4.24
query35 4.35 4.30 4.35
query36 1.10 1.06 1.06
query37 0.07 0.05 0.05
query38 0.04 0.03 0.03
query39 0.03 0.01 0.01
query40 0.16 0.13 0.13
query41 0.07 0.02 0.02
query42 0.03 0.02 0.01
query43 0.02 0.02 0.02
Total cold run time: 105.26 s
Total hot run time: 31.09 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1916623066
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 7387e2d101306fde0ae38f973fe1a21485a9b8da with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
Insert into select: 13.5 seconds inserted 10000000 Rows, about 740K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1928998026
<details>
<summary>TPC-H: <b>Total hot run time: 37321 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6fcf412f939e1dae00ac440b45ee6a5df9f40f3e, data reload: false
------ Round 1 ----------------------------------
q1 17672 4583 4549 4549
q2 2061 152 141 141
q3 10583 938 938 938
q4 4644 767 735 735
q5 7677 2871 2811 2811
q6 181 123 121 121
q7 1190 740 741 740
q8 9352 2029 2014 2014
q9 7275 6409 6356 6356
q10 8135 2479 2451 2451
q11 417 210 208 208
q12 768 273 287 273
q13 18028 3280 3277 3277
q14 284 242 250 242
q15 541 496 492 492
q16 480 404 425 404
q17 940 532 520 520
q18 6850 6130 5969 5969
q19 1554 1487 1387 1387
q20 620 359 324 324
q21 6735 3077 3090 3077
q22 792 292 297 292
Total cold run time: 106779 ms
Total hot run time: 37321 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4500 4453 4408 4408
q2 326 226 232 226
q3 2990 2829 2794 2794
q4 1897 1672 1750 1672
q5 5216 5274 5216 5216
q6 191 112 113 112
q7 2157 1798 1760 1760
q8 3085 3236 3236 3236
q9 8353 8315 8287 8287
q10 5772 3559 3540 3540
q11 534 464 462 462
q12 733 567 565 565
q13 11541 3082 3084 3082
q14 271 256 254 254
q15 530 496 493 493
q16 525 457 474 457
q17 1849 1720 1742 1720
q18 8031 7623 7453 7453
q19 10001 1539 1732 1539
q20 2146 1908 1912 1908
q21 4648 4493 4769 4493
q22 558 470 457 457
Total cold run time: 75854 ms
Total hot run time: 54134 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1915058502
<details>
<summary>ClickBench: <b>Total hot run time: 31 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit eb1dcc424240763422b59b495ddd719b15d44096, data reload: false
query1 0.03 0.03 0.03
query2 0.06 0.02 0.02
query3 0.23 0.06 0.06
query4 1.67 0.09 0.09
query5 0.52 0.51 0.52
query6 1.21 0.63 0.64
query7 0.02 0.02 0.01
query8 0.04 0.03 0.02
query9 0.55 0.50 0.50
query10 0.55 0.56 0.56
query11 0.11 0.09 0.09
query12 0.10 0.09 0.10
query13 0.61 0.61 0.60
query14 0.78 0.79 0.79
query15 0.80 0.77 0.77
query16 0.37 0.38 0.35
query17 1.05 1.03 1.02
query18 0.22 0.28 0.23
query19 1.84 1.77 1.78
query20 0.02 0.01 0.01
query21 15.41 0.60 0.60
query22 2.96 2.16 1.96
query23 17.33 0.92 0.76
query24 2.37 1.10 0.72
query25 0.41 0.21 0.10
query26 0.52 0.14 0.16
query27 0.05 0.05 0.06
query28 12.00 0.85 0.85
query29 12.51 3.41 3.28
query30 0.62 0.57 0.55
query31 2.80 0.34 0.37
query32 3.32 0.48 0.47
query33 3.19 3.20 3.21
query34 15.75 4.22 4.32
query35 4.38 4.33 4.27
query36 1.08 1.06 1.08
query37 0.06 0.05 0.05
query38 0.03 0.03 0.03
query39 0.02 0.01 0.02
query40 0.16 0.16 0.15
query41 0.07 0.02 0.02
query42 0.02 0.02 0.01
query43 0.02 0.02 0.01
Total cold run time: 105.86 s
Total hot run time: 31 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1916534776
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1915065009
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit eb1dcc424240763422b59b495ddd719b15d44096 with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.6 seconds inserted 10000000 Rows, about 735K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "TangSiyang2001 (via GitHub)" <gi...@apache.org>.
TangSiyang2001 commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1916515906
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "TangSiyang2001 (via GitHub)" <gi...@apache.org>.
TangSiyang2001 commented on PR #30536:
URL: https://github.com/apache/doris/pull/30536#issuecomment-1914921114
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](stmt-forward) record query result for proxy query to avoid EOF [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #30536:
URL: https://github.com/apache/doris/pull/30536
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org