You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "yiguolei (via GitHub)" <gi...@apache.org> on 2024/01/17 16:04:22 UTC
[PR] [improvement](fe) skip attach profile during send fragment [doris]
yiguolei opened a new pull request, #30085:
URL: https://github.com/apache/doris/pull/30085
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](fe) skip attach profile during send fragment [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30085:
URL: https://github.com/apache/doris/pull/30085#issuecomment-1896218819
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit eb0356da41ebd3f0e5fb5b2a7dfe2f3a5e3a922a with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
Insert into select: 14.6 seconds inserted 10000000 Rows, about 684K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](fe) skip attach profile during send fragment [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #30085:
URL: https://github.com/apache/doris/pull/30085
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](fe) skip attach profile during send fragment [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #30085:
URL: https://github.com/apache/doris/pull/30085#issuecomment-1896130920
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](fe) skip attach profile during send fragment [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30085:
URL: https://github.com/apache/doris/pull/30085#issuecomment-1897613497
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](fe) skip attach profile during send fragment [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30085:
URL: https://github.com/apache/doris/pull/30085#issuecomment-1897613530
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](fe) skip attach profile during send fragment [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #30085:
URL: https://github.com/apache/doris/pull/30085#issuecomment-1896120814
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](fe) skip attach profile during send fragment [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30085:
URL: https://github.com/apache/doris/pull/30085#issuecomment-1896203587
<details>
<summary>TPC-DS: <b>Total hot run time: 176759 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit eb0356da41ebd3f0e5fb5b2a7dfe2f3a5e3a922a, data reload: false
query1 955 328 330 328
query2 6566 1810 1961 1810
query3 6690 203 195 195
query4 31566 22148 22113 22113
query5 6882 529 523 523
query6 265 177 170 170
query7 4607 269 261 261
query8 224 170 180 170
query9 8359 2603 2567 2567
query10 417 217 246 217
query11 16671 15556 15427 15427
query12 119 67 66 66
query13 1682 383 377 377
query14 10598 6972 6971 6971
query15 211 182 181 181
query16 4605 232 238 232
query17 984 481 482 481
query18 1778 261 257 257
query19 175 134 134 134
query20 71 77 68 68
query21 193 132 133 132
query22 4917 4749 4793 4749
query23 31699 30853 30975 30853
query24 11571 2777 2832 2777
query25 570 308 298 298
query26 1294 149 142 142
query27 3220 270 270 270
query28 6884 1851 1831 1831
query29 1227 628 631 628
query30 278 136 143 136
query31 932 741 763 741
query32 84 49 49 49
query33 696 219 211 211
query34 1149 459 452 452
query35 872 788 769 769
query36 1352 1183 1207 1183
query37 90 56 57 56
query38 3400 3285 3236 3236
query39 1322 1287 1249 1249
query40 204 89 83 83
query41 41 35 34 34
query42 89 79 91 79
query43 497 478 475 475
query44 1068 681 701 681
query45 192 178 175 175
query46 1083 657 639 639
query47 1653 1523 1550 1523
query48 412 323 331 323
query49 1148 292 290 290
query50 685 301 312 301
query51 5400 5235 5263 5235
query52 95 76 80 76
query53 323 250 255 250
query54 889 439 448 439
query55 83 74 76 74
query56 174 165 171 165
query57 995 872 933 872
query58 184 160 179 160
query59 2916 2765 2596 2596
query60 212 185 187 185
query61 85 86 86 86
query62 621 360 376 360
query63 275 267 252 252
query64 4905 1833 1768 1768
query65 3337 3252 3239 3239
query66 1227 317 325 317
query67 15664 15059 15104 15059
query68 11957 528 540 528
query69 584 312 296 296
query70 1603 1529 1550 1529
query71 490 211 211 211
query72 4977 2787 2823 2787
query73 2695 311 313 311
query74 7042 6363 6472 6363
query75 4739 2263 2293 2263
query76 5735 1072 1026 1026
query77 690 240 260 240
query78 9247 8800 8639 8639
query79 980 497 530 497
query80 548 324 319 319
query81 468 203 208 203
query82 202 81 77 77
query83 143 122 118 118
query84 273 69 67 67
query85 1055 330 316 316
query86 389 400 382 382
query87 3486 3381 3290 3290
query88 2710 2180 2158 2158
query89 430 354 346 346
query90 1955 190 181 181
query91 166 121 134 121
query92 56 43 41 41
query93 1650 435 448 435
query94 1201 157 162 157
query95 508 456 464 456
query96 603 316 307 307
query97 4275 4165 4151 4151
query98 208 194 196 194
query99 1021 704 721 704
Total cold run time: 291718 ms
Total hot run time: 176759 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](fe) skip attach profile during send fragment [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30085:
URL: https://github.com/apache/doris/pull/30085#issuecomment-1896183513
<details>
<summary>TPC-H: <b>Total hot run time: 39131 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit eb0356da41ebd3f0e5fb5b2a7dfe2f3a5e3a922a, data reload: false
------ Round 1 ----------------------------------
q1 17936 5045 5145 5045
q2 2048 141 127 127
q3 10930 1173 1143 1143
q4 10397 793 815 793
q5 7774 3161 3161 3161
q6 196 124 121 121
q7 865 505 491 491
q8 9257 1963 1937 1937
q9 7320 6431 6420 6420
q10 8243 3080 3072 3072
q11 417 208 209 208
q12 352 191 189 189
q13 18213 3638 3550 3550
q14 272 251 229 229
q15 548 530 521 521
q16 506 416 418 416
q17 976 515 543 515
q18 7835 7356 6873 6873
q19 4918 1345 1432 1345
q20 620 313 290 290
q21 2792 2448 2385 2385
q22 383 300 317 300
Total cold run time: 112798 ms
Total hot run time: 39131 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5510 5205 5225 5205
q2 326 221 213 213
q3 3277 3168 3242 3168
q4 2098 2088 2038 2038
q5 5949 5811 5936 5811
q6 197 115 115 115
q7 2264 1872 1880 1872
q8 3197 3347 3374 3347
q9 9059 8903 8758 8758
q10 3923 3818 3842 3818
q11 554 459 444 444
q12 764 660 633 633
q13 16963 3147 3129 3129
q14 295 255 272 255
q15 543 506 511 506
q16 521 469 486 469
q17 1869 1820 1829 1820
q18 9448 16928 9663 9663
q19 26591 1632 1536 1536
q20 4625 1936 1926 1926
q21 14436 5284 5325 5284
q22 971 558 523 523
Total cold run time: 113380 ms
Total hot run time: 60533 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](fe) skip attach profile during send fragment [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30085:
URL: https://github.com/apache/doris/pull/30085#issuecomment-1896213267
<details>
<summary>ClickBench: <b>Total hot run time: 30.84 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit eb0356da41ebd3f0e5fb5b2a7dfe2f3a5e3a922a, data reload: false
query1 0.03 0.02 0.02
query2 0.06 0.02 0.02
query3 0.23 0.05 0.04
query4 1.67 0.07 0.07
query5 0.52 0.52 0.52
query6 1.29 0.64 0.62
query7 0.02 0.01 0.01
query8 0.03 0.02 0.02
query9 0.54 0.51 0.49
query10 0.57 0.57 0.56
query11 0.12 0.09 0.09
query12 0.12 0.09 0.08
query13 0.60 0.60 0.59
query14 0.77 0.79 0.77
query15 0.80 0.78 0.78
query16 0.37 0.36 0.37
query17 1.00 1.01 1.01
query18 0.24 0.23 0.27
query19 1.82 1.77 1.75
query20 0.01 0.01 0.01
query21 15.41 0.56 0.55
query22 2.54 2.32 2.05
query23 17.10 0.90 0.83
query24 2.49 1.52 0.91
query25 0.54 0.26 0.15
query26 0.50 0.13 0.13
query27 0.05 0.06 0.06
query28 11.86 0.76 0.75
query29 12.49 3.12 3.15
query30 0.57 0.51 0.49
query31 2.78 0.34 0.34
query32 3.36 0.49 0.48
query33 3.22 3.22 3.29
query34 15.76 4.21 4.19
query35 4.33 4.30 4.20
query36 1.11 1.08 1.07
query37 0.07 0.05 0.05
query38 0.03 0.03 0.03
query39 0.03 0.01 0.01
query40 0.17 0.13 0.13
query41 0.07 0.01 0.02
query42 0.03 0.01 0.01
query43 0.03 0.02 0.02
Total cold run time: 105.35 s
Total hot run time: 30.84 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org