You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "HHoflittlefish777 (via GitHub)" <gi...@apache.org> on 2024/01/01 14:20:24 UTC
[PR] Revert "[improve](move-memtable) cancel load rapidly when stream close wait (#29322)" [doris]
HHoflittlefish777 opened a new pull request, #29371:
URL: https://github.com/apache/doris/pull/29371
## Proposed changes
This reverts commit bbf58c5aa42d40e66bc6ccc9ed91a4fcb4bdfff7.
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Revert "[improve](move-memtable) cancel load rapidly when stream close wait (#29322)" [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29371:
URL: https://github.com/apache/doris/pull/29371#issuecomment-1873349757
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Revert "[improve](move-memtable) cancel load rapidly when stream close wait (#29322)" [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29371:
URL: https://github.com/apache/doris/pull/29371#issuecomment-1873347972
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Revert "[improve](move-memtable) cancel load rapidly when stream close wait (#29322)" [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29371:
URL: https://github.com/apache/doris/pull/29371#issuecomment-1873347956
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Revert "[improve](move-memtable) cancel load rapidly when stream close wait (#29322)" [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #29371:
URL: https://github.com/apache/doris/pull/29371
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Revert "[improve](move-memtable) cancel load rapidly when stream close wait (#29322)" [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29371:
URL: https://github.com/apache/doris/pull/29371#issuecomment-1873358479
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.67 seconds
stream load tsv: 564 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.4 seconds inserted 10000000 Rows, about 352K ops/s
storage size: 17183519098 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Revert "[improve](move-memtable) cancel load rapidly when stream close wait (#29322)" [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29371:
URL: https://github.com/apache/doris/pull/29371#issuecomment-1873354286
TeamCity be ut coverage result:
Function Coverage: 36.62% (8611/23513)
Line Coverage: 28.68% (69984/244032)
Region Coverage: 27.67% (36224/130929)
Branch Coverage: 24.36% (18507/75958)
Coverage Report: http://coverage.selectdb-in.cc/coverage/ccd8824ba3dd2e4e6967dde6a686a0c815370c24_ccd8824ba3dd2e4e6967dde6a686a0c815370c24/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Revert "[improve](move-memtable) cancel load rapidly when stream close wait (#29322)" [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29371:
URL: https://github.com/apache/doris/pull/29371#issuecomment-1873363238
TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools
```
TPC-DS sf100 test result on commit ccd8824ba3dd2e4e6967dde6a686a0c815370c24, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 942 359 339 339
query2 6426 2126 1969 1969
query3 6658 205 199 199
query4 28078 22390 22518 22390
query5 4921 575 541 541
query6 287 189 180 180
query7 4583 272 274 272
query8 234 201 199 199
query9 8267 2649 2742 2649
query10 456 238 264 238
query11 16224 15517 15586 15517
query12 135 79 77 77
query13 1654 326 327 326
query14 12227 7066 7064 7064
query15 249 188 191 188
query16 6510 278 275 275
query17 1807 512 490 490
query18 1929 270 264 264
query19 246 150 143 143
query20 83 78 79 78
query21 183 97 98 97
query22 5029 4812 4421 4421
query23 32004 31068 31137 31068
query24 10866 2815 2821 2815
query25 596 349 356 349
query26 1520 143 143 143
query27 2836 278 278 278
query28 6979 1941 1947 1941
query29 1560 381 402 381
query30 277 140 148 140
query31 969 767 781 767
query32 87 60 57 57
query33 715 278 269 269
query34 852 444 440 440
query35 926 793 759 759
query36 1292 1293 1252 1252
query37 108 74 75 74
query38 3384 3282 3281 3281
query39 1315 1285 1288 1285
query40 192 95 90 90
query41 36 35 35 35
query42 109 88 94 88
query43 524 479 496 479
query44 1074 710 727 710
query45 201 188 192 188
query46 1064 644 651 644
query47 1644 1559 1606 1559
query48 328 267 262 262
query49 1084 325 329 325
query50 790 363 324 324
query51 5316 5258 5226 5226
query52 99 86 89 86
query53 213 151 148 148
query54 1423 557 586 557
query55 102 90 89 89
query56 210 210 200 200
query57 998 956 953 953
query58 214 210 205 205
query59 2914 2719 2713 2713
query60 249 252 229 229
query61 91 90 90 90
query62 615 453 473 453
query63 174 154 146 146
query64 5025 1793 1761 1761
query65 3335 3265 3290 3265
query66 1204 341 334 334
query67 15873 15508 15335 15335
query68 12844 544 555 544
query69 507 254 254 254
query70 1711 1535 1531 1531
query71 493 231 228 228
query72 5795 3662 3659 3659
query73 2967 321 308 308
query74 6993 6389 6445 6389
query75 5416 2268 2283 2268
query76 6310 1141 1119 1119
query77 707 247 249 247
query78 9100 8911 8625 8625
query79 1039 535 498 498
query80 697 376 366 366
query81 466 211 215 211
query82 207 101 102 101
query83 162 143 135 135
query84 243 53 53 53
query85 929 290 289 289
query86 389 386 371 371
query87 3650 3345 3370 3345
query88 2816 2225 2245 2225
query89 345 268 257 257
query90 1833 211 210 210
query91 128 99 92 92
query92 62 55 53 53
query93 1428 510 444 444
query94 793 192 189 189
query95 461 410 404 404
query96 624 314 313 313
query97 4264 4165 4141 4141
query98 216 208 194 194
query99 1081 810 833 810
Total cold run time: 293191 ms
Total hot run time: 179582 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Revert "[improve](move-memtable) cancel load rapidly when stream close wait (#29322)" [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29371:
URL: https://github.com/apache/doris/pull/29371#issuecomment-1873354165
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
```
Tpch sf100 test result on commit ccd8824ba3dd2e4e6967dde6a686a0c815370c24, data reload: false
run tpch-sf100 query with default conf and session variables
q1 5502 5054 5107 5054
q2 389 168 157 157
q3 1432 1162 1169 1162
q4 1089 803 822 803
q5 3111 3089 3103 3089
q6 227 136 129 129
q7 980 542 540 540
q8 2137 2158 2273 2158
q9 6656 6670 6627 6627
q10 3161 3163 3177 3163
q11 347 223 226 223
q12 379 240 232 232
q13 4385 3648 3664 3648
q14 244 225 204 204
q15 612 557 535 535
q16 468 415 413 413
q17 1031 541 476 476
q18 7067 6676 6830 6676
q19 1632 1500 1599 1500
q20 571 332 332 332
q21 2843 2388 2448 2388
q22 398 334 331 331
Total cold run time: 44661 ms
Total hot run time: 39840 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 5144 5119 5106 5106
q2 336 252 235 235
q3 3368 3308 3293 3293
q4 2136 2021 2022 2021
q5 5923 5911 5906 5906
q6 225 125 121 121
q7 2346 1883 1913 1883
q8 3545 3639 3679 3639
q9 9046 8975 8975 8975
q10 3860 3882 3929 3882
q11 581 467 487 467
q12 818 669 657 657
q13 3912 3204 3221 3204
q14 300 280 271 271
q15 610 546 542 542
q16 556 535 498 498
q17 2033 1810 1803 1803
q18 8798 8289 8357 8289
q19 1726 1692 1725 1692
q20 2295 2000 2001 2000
q21 5801 5406 5334 5334
q22 557 498 480 480
Total cold run time: 63916 ms
Total hot run time: 60298 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Revert "[improve](move-memtable) cancel load rapidly when stream close wait (#29322)" [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29371:
URL: https://github.com/apache/doris/pull/29371#issuecomment-1873358902
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
```
Tpch sf100 test result on commit ccd8824ba3dd2e4e6967dde6a686a0c815370c24, data reload: false
------ Round 1 ----------------------------------
q1 17636 5185 5122 5122
q2 2019 158 141 141
q3 10519 1075 1131 1075
q4 10177 797 778 778
q5 7789 2833 2939 2833
q6 208 132 134 132
q7 910 557 499 499
q8 9297 2017 2042 2017
q9 6839 6382 6355 6355
q10 8232 3055 2985 2985
q11 436 222 222 222
q12 396 233 244 233
q13 17989 3656 3648 3648
q14 239 220 220 220
q15 574 564 531 531
q16 470 395 393 393
q17 957 444 519 444
q18 7262 6701 6666 6666
q19 1578 1371 1371 1371
q20 691 351 333 333
q21 2808 2355 2487 2355
q22 381 315 323 315
Total cold run time: 107407 ms
Total hot run time: 38668 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5115 5120 5091 5091
q2 340 236 258 236
q3 3286 3281 3240 3240
q4 2112 2024 1989 1989
q5 5792 5756 5766 5756
q6 219 123 130 123
q7 2301 1890 1887 1887
q8 3360 3432 3471 3432
q9 8814 8778 8702 8702
q10 3810 3885 3864 3864
q11 580 475 481 475
q12 807 661 643 643
q13 6743 3262 3243 3243
q14 293 272 267 267
q15 606 545 528 528
q16 531 513 487 487
q17 1943 1773 1775 1773
q18 8622 8495 8243 8243
q19 1633 1604 1576 1576
q20 2195 1970 1946 1946
q21 5540 5218 5204 5204
q22 534 514 484 484
Total cold run time: 65176 ms
Total hot run time: 59189 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] Revert "[improve](move-memtable) cancel load rapidly when stream close wait (#29322)" [doris]
Posted by "HHoflittlefish777 (via GitHub)" <gi...@apache.org>.
HHoflittlefish777 commented on PR #29371:
URL: https://github.com/apache/doris/pull/29371#issuecomment-1873347335
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org