You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Gabriel39 (via GitHub)" <gi...@apache.org> on 2024/01/10 03:16:50 UTC
[PR] [pipelineX](bug) Fix correctness problem using multiple BE [doris]
Gabriel39 opened a new pull request, #29765:
URL: https://github.com/apache/doris/pull/29765
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) Fix correctness problem using multiple BE [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29765:
URL: https://github.com/apache/doris/pull/29765#issuecomment-1884273692
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) Fix correctness problem using multiple BE [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 merged PR #29765:
URL: https://github.com/apache/doris/pull/29765
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) Fix correctness problem using multiple BE [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #29765:
URL: https://github.com/apache/doris/pull/29765#issuecomment-1884151166
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) Fix correctness problem using multiple BE [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29765:
URL: https://github.com/apache/doris/pull/29765#issuecomment-1884178661
<details>
<summary>TPC-DS: <b>Total hot run time: 178540 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3bef13843b47c668fbec5619b8ad5315dc98999e, data reload: false
query1 934 341 336 336
query2 6693 1830 1879 1830
query3 6719 217 207 207
query4 25887 22297 22165 22165
query5 5800 540 516 516
query6 268 187 176 176
query7 4610 284 271 271
query8 223 218 203 203
query9 9155 2710 2777 2710
query10 467 250 243 243
query11 16426 15339 15226 15226
query12 134 84 79 79
query13 1636 342 333 333
query14 12993 7336 7463 7336
query15 238 209 210 209
query16 6518 280 283 280
query17 1994 504 481 481
query18 1953 275 266 266
query19 292 144 141 141
query20 87 86 77 77
query21 192 91 89 89
query22 4946 4888 4812 4812
query23 32168 31241 31089 31089
query24 11307 2878 2801 2801
query25 581 346 342 342
query26 1574 157 158 157
query27 2828 281 275 275
query28 7200 1889 1904 1889
query29 1608 407 407 407
query30 279 148 145 145
query31 998 781 772 772
query32 91 66 58 58
query33 729 263 270 263
query34 920 440 437 437
query35 908 726 718 718
query36 1439 1292 1240 1240
query37 101 74 67 67
query38 3427 3296 3308 3296
query39 1325 1288 1275 1275
query40 297 98 93 93
query41 40 35 36 35
query42 102 92 94 92
query43 577 504 500 500
query44 1089 712 719 712
query45 207 191 182 182
query46 1139 675 650 650
query47 1685 1539 1614 1539
query48 411 324 327 324
query49 1229 319 320 319
query50 858 369 353 353
query51 5330 5247 5275 5247
query52 94 88 85 85
query53 229 157 156 156
query54 1367 598 578 578
query55 100 95 94 94
query56 225 204 200 200
query57 1035 934 934 934
query58 235 217 221 217
query59 2741 2450 2458 2450
query60 256 250 230 230
query61 81 80 79 79
query62 651 469 462 462
query63 174 151 153 151
query64 5825 1697 1654 1654
query65 3369 3278 3270 3270
query66 1141 337 328 328
query67 15645 15436 15225 15225
query68 13184 531 526 526
query69 582 277 285 277
query70 1925 1514 1552 1514
query71 509 232 231 231
query72 4904 2824 2812 2812
query73 3421 329 338 329
query74 7006 6475 6409 6409
query75 5359 2303 2250 2250
query76 6437 1193 1258 1193
query77 708 271 300 271
query78 10012 8572 8434 8434
query79 1241 520 503 503
query80 533 368 350 350
query81 421 211 211 211
query82 211 92 89 89
query83 187 136 143 136
query84 267 57 54 54
query85 725 291 277 277
query86 392 377 381 377
query87 3592 3402 3406 3402
query88 2916 2299 2327 2299
query89 341 253 267 253
query90 1813 205 201 201
query91 168 132 131 131
query92 60 56 54 54
query93 1289 486 436 436
query94 846 195 194 194
query95 485 434 427 427
query96 692 331 326 326
query97 4304 4190 4149 4149
query98 217 193 189 189
query99 1105 868 860 860
Total cold run time: 297632 ms
Total hot run time: 178540 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) Fix correctness problem using multiple BE [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29765:
URL: https://github.com/apache/doris/pull/29765#issuecomment-1884182242
<details>
<summary>ClickBench: <b>Total hot run time: 30.18 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3bef13843b47c668fbec5619b8ad5315dc98999e, data reload: false
query1 0.08 0.06 0.06
query2 0.06 0.02 0.02
query3 0.25 0.12 0.11
query4 1.78 0.13 0.12
query5 0.53 0.51 0.51
query6 1.42 0.64 0.63
query7 0.02 0.02 0.01
query8 0.04 0.03 0.02
query9 0.55 0.50 0.50
query10 0.54 0.54 0.58
query11 0.13 0.10 0.09
query12 0.12 0.09 0.09
query13 0.60 0.60 0.60
query14 0.79 0.79 0.79
query15 0.82 0.80 0.80
query16 0.35 0.34 0.35
query17 0.96 0.98 0.99
query18 0.25 0.26 0.24
query19 1.83 1.73 1.75
query20 0.01 0.01 0.01
query21 15.41 0.56 0.59
query22 2.64 2.37 1.80
query23 17.34 0.76 0.76
query24 5.38 0.25 0.89
query25 1.97 0.19 0.15
query26 0.14 0.15 0.14
query27 0.15 0.15 0.14
query28 10.69 0.84 0.82
query29 12.57 3.43 3.25
query30 0.52 0.50 0.49
query31 2.77 0.38 0.36
query32 3.35 0.50 0.49
query33 3.24 3.21 3.21
query34 16.06 4.14 4.14
query35 4.17 4.16 4.15
query36 1.12 1.05 1.04
query37 0.06 0.05 0.04
query38 0.03 0.03 0.03
query39 0.02 0.01 0.02
query40 0.16 0.13 0.13
query41 0.07 0.02 0.01
query42 0.02 0.01 0.02
query43 0.03 0.02 0.01
Total cold run time: 109.04 s
Total hot run time: 30.18 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) Fix correctness problem using multiple BE [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29765:
URL: https://github.com/apache/doris/pull/29765#issuecomment-1884184267
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 3bef13843b47c668fbec5619b8ad5315dc98999e with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 14.2 seconds inserted 10000000 Rows, about 704K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) Fix correctness problem using multiple BE [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29765:
URL: https://github.com/apache/doris/pull/29765#issuecomment-1884273648
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) Fix correctness problem using multiple BE [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29765:
URL: https://github.com/apache/doris/pull/29765#issuecomment-1884171563
<details>
<summary>TPC-H: <b>Total hot run time: 38429 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3bef13843b47c668fbec5619b8ad5315dc98999e, data reload: false
------ Round 1 ----------------------------------
q1 17947 4985 4951 4951
q2 2053 146 135 135
q3 10684 1121 1189 1121
q4 10411 792 834 792
q5 7888 3053 3108 3053
q6 206 121 126 121
q7 867 500 480 480
q8 9290 1941 2006 1941
q9 6542 6437 6435 6435
q10 8203 3054 2999 2999
q11 413 197 217 197
q12 363 191 189 189
q13 18129 3431 3431 3431
q14 241 210 215 210
q15 553 517 508 508
q16 432 376 395 376
q17 936 562 560 560
q18 7317 6662 6619 6619
q19 1572 1345 1356 1345
q20 596 318 311 311
q21 2786 2344 2401 2344
q22 361 334 311 311
Total cold run time: 107790 ms
Total hot run time: 38429 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5010 5004 4956 4956
q2 308 196 197 196
q3 3307 3304 3285 3285
q4 2216 2233 2214 2214
q5 5771 5760 5757 5757
q6 191 115 117 115
q7 2304 1835 1835 1835
q8 3444 3556 3535 3535
q9 8850 8785 8793 8785
q10 3722 3800 3818 3800
q11 563 428 429 428
q12 767 585 612 585
q13 4031 3174 3194 3174
q14 289 262 259 259
q15 567 506 507 506
q16 511 462 469 462
q17 2055 2022 1994 1994
q18 8743 8353 8388 8353
q19 1602 1579 1584 1579
q20 2142 1913 1916 1913
q21 5983 5628 5682 5628
q22 575 507 489 489
Total cold run time: 62951 ms
Total hot run time: 59848 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) Fix correctness problem using multiple BE [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29765:
URL: https://github.com/apache/doris/pull/29765#issuecomment-1884172638
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.09 seconds
stream load tsv: 566 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.2 seconds inserted 10000000 Rows, about 471K ops/s
storage size: 17183905520 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org