You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "airborne12 (via GitHub)" <gi...@apache.org> on 2024/09/10 01:58:06 UTC
[PR] [Fix](inverted index) gc TEMP colum when next_batch in segment iterator #40563 [doris]
airborne12 opened a new pull request, #40575:
URL: https://github.com/apache/doris/pull/40575
cherry pick from #40563
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](inverted index) gc TEMP colum when next_batch in segment iterator #40563 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40575:
URL: https://github.com/apache/doris/pull/40575#issuecomment-2339489499
<details>
<summary>TPC-H: <b>Total hot run time: 49044 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 352539b64b29abd0663f1ae5de191f1d1832035e, data reload: false
------ Round 1 ----------------------------------
q1 17566 4368 4326 4326
q2 2070 149 144 144
q3 10469 1882 1958 1882
q4 10280 1269 1308 1269
q5 8528 3837 3882 3837
q6 229 123 127 123
q7 2005 1611 1591 1591
q8 9280 2735 2693 2693
q9 10064 9794 9828 9794
q10 8596 3526 3531 3526
q11 417 240 251 240
q12 467 303 289 289
q13 18386 3927 4014 3927
q14 359 329 340 329
q15 508 455 458 455
q16 543 458 443 443
q17 1116 955 947 947
q18 7305 6955 6963 6955
q19 1676 1556 1463 1463
q20 536 298 318 298
q21 4410 4132 4122 4122
q22 502 391 404 391
Total cold run time: 115312 ms
Total hot run time: 49044 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4356 4298 4325 4298
q2 321 235 215 215
q3 4137 4121 4108 4108
q4 2738 2739 2752 2739
q5 7112 7147 7068 7068
q6 238 116 119 116
q7 3220 2859 2859 2859
q8 4352 4435 4458 4435
q9 13682 13686 13619 13619
q10 4290 4317 4273 4273
q11 738 671 676 671
q12 1025 855 866 855
q13 8145 3784 3775 3775
q14 460 425 421 421
q15 498 462 452 452
q16 622 594 597 594
q17 3794 3822 3798 3798
q18 8885 8810 8823 8810
q19 1754 1650 1650 1650
q20 2389 2144 2123 2123
q21 8506 8708 8715 8708
q22 1130 1001 996 996
Total cold run time: 82392 ms
Total hot run time: 76583 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](inverted index) gc TEMP colum when next_batch in segment iterator #40563 [doris]
Posted by "airborne12 (via GitHub)" <gi...@apache.org>.
airborne12 commented on PR #40575:
URL: https://github.com/apache/doris/pull/40575#issuecomment-2339459225
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](inverted index) gc TEMP colum when next_batch in segment iterator #40563 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40575:
URL: https://github.com/apache/doris/pull/40575#issuecomment-2339520773
TeamCity be ut coverage result:
Function Coverage: 37.85% (8139/21503)
Line Coverage: 29.58% (66980/226409)
Region Coverage: 29.07% (34555/118869)
Branch Coverage: 24.98% (17808/71284)
Coverage Report: http://coverage.selectdb-in.cc/coverage/352539b64b29abd0663f1ae5de191f1d1832035e_352539b64b29abd0663f1ae5de191f1d1832035e/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](inverted index) gc TEMP colum when next_batch in segment iterator #40563 [doris]
Posted by "airborne12 (via GitHub)" <gi...@apache.org>.
airborne12 commented on PR #40575:
URL: https://github.com/apache/doris/pull/40575#issuecomment-2339520863
run p1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](inverted index) gc TEMP colum when next_batch in segment iterator #40563 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40575:
URL: https://github.com/apache/doris/pull/40575#issuecomment-2339500445
<details>
<summary>TPC-DS: <b>Total hot run time: 211842 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 352539b64b29abd0663f1ae5de191f1d1832035e, data reload: false
query1 2804 429 398 398
query2 9553 2224 2059 2059
query3 6913 209 206 206
query4 23218 21872 21560 21560
query5 19749 6501 6594 6501
query6 386 219 225 219
query7 5080 314 321 314
query8 263 252 262 252
query9 3059 2623 2558 2558
query10 479 304 296 296
query11 15507 15235 15294 15235
query12 126 80 72 72
query13 1004 432 436 432
query14 18582 13864 13210 13210
query15 392 218 227 218
query16 6442 285 262 262
query17 1586 946 904 904
query18 887 322 308 308
query19 214 157 156 156
query20 85 78 80 78
query21 189 102 97 97
query22 5270 4959 5005 4959
query23 34178 33304 33568 33304
query24 6556 6320 6275 6275
query25 525 426 422 422
query26 1008 163 161 161
query27 2274 304 300 300
query28 6069 2267 2206 2206
query29 2973 2704 2731 2704
query30 242 175 168 168
query31 921 771 722 722
query32 67 62 60 60
query33 437 265 264 264
query34 875 477 481 477
query35 1101 925 923 923
query36 1251 1472 1408 1408
query37 90 59 60 59
query38 3082 2879 2856 2856
query39 1367 1313 1314 1313
query40 211 94 92 92
query41 39 36 36 36
query42 93 80 93 80
query43 591 626 605 605
query44 1164 720 728 720
query45 242 230 231 230
query46 1356 965 965 965
query47 1886 1700 1767 1700
query48 512 406 413 406
query49 611 374 367 367
query50 865 629 585 585
query51 4889 4685 4671 4671
query52 84 80 81 80
query53 240 184 176 176
query54 2674 2490 2498 2490
query55 99 88 86 86
query56 244 212 207 207
query57 1226 1091 1231 1091
query58 220 205 221 205
query59 3556 3232 3384 3232
query60 219 203 196 196
query61 99 95 98 95
query62 810 439 457 439
query63 202 175 181 175
query64 3370 1580 1505 1505
query65 3614 3513 3577 3513
query66 781 422 451 422
query67 17042 15268 15635 15268
query68 8996 659 665 659
query69 498 261 254 254
query70 1528 1293 1408 1293
query71 407 311 310 310
query72 6845 4855 4772 4772
query73 751 330 322 322
query74 6288 5735 5873 5735
query75 4613 3640 3726 3640
query76 4628 1124 1208 1124
query77 582 256 254 254
query78 12547 11804 11595 11595
query79 6117 641 641 641
query80 2278 388 375 375
query81 511 244 233 233
query82 1616 95 97 95
query83 173 135 136 135
query84 261 73 69 69
query85 1373 332 320 320
query86 361 308 296 296
query87 3303 3045 2959 2959
query88 4863 2380 2386 2380
query89 401 288 292 288
query90 1759 208 216 208
query91 157 134 128 128
query92 59 53 50 50
query93 4791 612 587 587
query94 900 207 211 207
query95 2069 1928 1623 1623
query96 656 337 330 330
query97 6431 6334 6468 6334
query98 242 217 205 205
query99 2862 868 911 868
Total cold run time: 320609 ms
Total hot run time: 211842 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](inverted index) gc TEMP colum when next_batch in segment iterator #40563 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40575:
URL: https://github.com/apache/doris/pull/40575#issuecomment-2339459284
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](inverted index) gc TEMP colum when next_batch in segment iterator #40563 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40575:
URL: https://github.com/apache/doris/pull/40575#issuecomment-2339507633
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 352539b64b29abd0663f1ae5de191f1d1832035e with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 21.6 seconds inserted 10000000 Rows, about 462K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](inverted index) gc TEMP colum when next_batch in segment iterator #40563 [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40575:
URL: https://github.com/apache/doris/pull/40575#issuecomment-2339467528
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](inverted index) gc TEMP colum when next_batch in segment iterator #40563 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40575:
URL: https://github.com/apache/doris/pull/40575#issuecomment-2339505002
<details>
<summary>ClickBench: <b>Total hot run time: 30.95 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 352539b64b29abd0663f1ae5de191f1d1832035e, data reload: false
query1 0.02 0.02 0.02
query2 0.06 0.03 0.03
query3 0.24 0.04 0.04
query4 1.80 0.06 0.06
query5 0.53 0.53 0.52
query6 1.31 0.61 0.62
query7 0.01 0.01 0.01
query8 0.03 0.02 0.02
query9 0.52 0.49 0.47
query10 0.53 0.54 0.53
query11 0.11 0.09 0.09
query12 0.12 0.09 0.09
query13 0.62 0.62 0.61
query14 0.78 0.80 0.78
query15 0.78 0.76 0.75
query16 0.38 0.38 0.36
query17 1.00 1.01 1.01
query18 0.21 0.28 0.22
query19 1.96 1.84 1.84
query20 0.01 0.00 0.00
query21 15.45 0.56 0.54
query22 1.94 2.02 1.68
query23 17.26 0.97 0.91
query24 4.25 3.47 1.16
query25 0.36 0.16 0.04
query26 0.75 0.17 0.15
query27 0.04 0.04 0.03
query28 6.25 0.77 0.74
query29 12.76 2.35 2.33
query30 0.54 0.54 0.54
query31 2.81 0.38 0.38
query32 3.38 0.49 0.51
query33 3.06 3.05 3.05
query34 15.28 4.80 4.76
query35 4.83 4.85 4.82
query36 1.06 1.01 1.02
query37 0.05 0.05 0.05
query38 0.03 0.02 0.03
query39 0.02 0.01 0.02
query40 0.15 0.14 0.14
query41 0.07 0.01 0.01
query42 0.02 0.01 0.02
query43 0.02 0.02 0.02
Total cold run time: 101.4 s
Total hot run time: 30.95 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](inverted index) gc TEMP colum when next_batch in segment iterator #40563 [doris]
Posted by "airborne12 (via GitHub)" <gi...@apache.org>.
airborne12 merged PR #40575:
URL: https://github.com/apache/doris/pull/40575
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org