You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "airborne12 (via GitHub)" <gi...@apache.org> on 2024/01/02 10:17:43 UTC
[PR] [Enhancement](inverted index) strictly checkout inverted index properties [doris]
airborne12 opened a new pull request, #29421:
URL: https://github.com/apache/doris/pull/29421
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](inverted index) strictly checkout inverted index properties [doris]
Posted by "qidaye (via GitHub)" <gi...@apache.org>.
qidaye commented on PR #29421:
URL: https://github.com/apache/doris/pull/29421#issuecomment-1873941926
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](inverted index) strictly checkout inverted index properties [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29421:
URL: https://github.com/apache/doris/pull/29421#issuecomment-1873847495
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](inverted index) strictly checkout inverted index properties [doris]
Posted by "eldenmoon (via GitHub)" <gi...@apache.org>.
eldenmoon merged PR #29421:
URL: https://github.com/apache/doris/pull/29421
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](inverted index) strictly checkout inverted index properties [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29421:
URL: https://github.com/apache/doris/pull/29421#issuecomment-1873847546
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](inverted index) strictly checkout inverted index properties [doris]
Posted by "airborne12 (via GitHub)" <gi...@apache.org>.
airborne12 commented on PR #29421:
URL: https://github.com/apache/doris/pull/29421#issuecomment-1873959480
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](inverted index) strictly checkout inverted index properties [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29421:
URL: https://github.com/apache/doris/pull/29421#issuecomment-1873978952
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
```
Tpch sf100 test result on commit 3c8207fd3bac9e3582471f82785a87eada114150, data reload: false
run tpch-sf100 query with default conf and session variables
q1 5518 5084 5067 5067
q2 409 167 158 158
q3 1465 1137 1094 1094
q4 1098 802 859 802
q5 3122 3121 3147 3121
q6 228 136 136 136
q7 981 551 519 519
q8 2165 2219 2272 2219
q9 6731 6644 6648 6644
q10 3185 3084 3120 3084
q11 343 216 213 213
q12 378 231 234 231
q13 4389 3641 3637 3637
q14 257 213 213 213
q15 605 550 543 543
q16 460 433 397 397
q17 1051 485 487 485
q18 7120 6795 6733 6733
q19 1651 1548 1537 1537
q20 598 335 342 335
q21 2879 2481 2495 2481
q22 407 332 338 332
Total cold run time: 45040 ms
Total hot run time: 39981 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 5141 5106 5101 5101
q2 348 251 261 251
q3 3384 3327 3306 3306
q4 2136 2037 1967 1967
q5 5972 5931 5938 5931
q6 229 123 124 123
q7 2374 1908 1969 1908
q8 3591 3699 3657 3657
q9 9069 8942 8946 8942
q10 3887 3919 3885 3885
q11 586 484 469 469
q12 805 677 691 677
q13 3895 3181 3201 3181
q14 298 286 270 270
q15 616 541 558 541
q16 570 487 530 487
q17 2020 1802 1826 1802
q18 8764 8380 8407 8380
q19 1761 1750 1676 1676
q20 2304 1999 1980 1980
q21 5690 5347 5253 5253
q22 570 466 489 466
Total cold run time: 64010 ms
Total hot run time: 60253 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](inverted index) strictly checkout inverted index properties [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29421:
URL: https://github.com/apache/doris/pull/29421#issuecomment-1873989143
TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools
```
TPC-DS sf100 test result on commit 3c8207fd3bac9e3582471f82785a87eada114150, data reload: false
run tpcds-sf100 query with default conf and session variables
query1 931 354 346 346
query2 6465 2099 1922 1922
query3 6648 202 205 202
query4 24371 22498 22468 22468
query5 5197 519 574 519
query6 282 181 186 181
query7 4518 289 280 280
query8 257 216 212 212
query9 8255 2580 2652 2580
query10 375 246 272 246
query11 16239 15623 15589 15589
query12 130 79 74 74
query13 1539 327 335 327
query14 11328 7336 7384 7336
query15 360 216 217 216
query16 6428 270 269 269
query17 1778 520 521 520
query18 1936 266 260 260
query19 287 160 163 160
query20 83 77 81 77
query21 192 97 104 97
query22 4945 4889 4844 4844
query23 32371 31346 31546 31346
query24 12412 3498 3472 3472
query25 591 377 366 366
query26 1496 157 154 154
query27 2728 299 304 299
query28 6449 1976 1956 1956
query29 1681 437 405 405
query30 302 148 151 148
query31 1096 790 816 790
query32 93 67 57 57
query33 695 286 285 285
query34 845 458 438 438
query35 943 803 758 758
query36 1301 1212 1266 1212
query37 183 75 73 73
query38 3386 3305 3331 3305
query39 1362 1310 1306 1306
query40 302 95 95 95
query41 36 35 34 34
query42 93 93 91 91
query43 539 498 511 498
query44 1116 708 728 708
query45 232 228 218 218
query46 1175 788 780 780
query47 1675 1580 1519 1519
query48 354 281 292 281
query49 1209 329 323 323
query50 1138 834 858 834
query51 5377 5366 5309 5309
query52 93 86 86 86
query53 209 174 180 174
query54 1325 600 619 600
query55 89 89 101 89
query56 217 193 203 193
query57 1060 948 961 948
query58 232 211 215 211
query59 2698 2557 2589 2557
query60 259 248 244 244
query61 82 87 81 81
query62 1379 514 503 503
query63 189 170 173 170
query64 6077 1811 1771 1771
query65 3366 3315 3295 3295
query66 1055 340 336 336
query67 15568 15230 15574 15230
query68 10146 524 547 524
query69 507 255 253 253
query70 1750 1561 1551 1551
query71 505 231 225 225
query72 6197 4172 4042 4042
query73 2106 318 312 312
query74 6903 6479 6482 6479
query75 4981 2281 2306 2281
query76 6230 1081 1134 1081
query77 662 298 287 287
query78 9124 8862 8535 8535
query79 1257 532 542 532
query80 564 371 385 371
query81 471 214 218 214
query82 217 103 98 98
query83 163 134 139 134
query84 246 53 53 53
query85 872 288 279 279
query86 401 386 395 386
query87 3595 3443 3437 3437
query88 3510 2302 2297 2297
query89 374 273 293 273
query90 1701 211 222 211
query91 122 89 92 89
query92 59 52 53 52
query93 3636 509 488 488
query94 722 188 192 188
query95 460 409 418 409
query96 649 322 315 315
query97 4280 4127 4217 4127
query98 212 195 188 188
query99 1696 952 879 879
Total cold run time: 291940 ms
Total hot run time: 182863 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](inverted index) strictly checkout inverted index properties [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29421:
URL: https://github.com/apache/doris/pull/29421#issuecomment-1873977935
TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools
```
Tpch sf100 test result on commit 3c8207fd3bac9e3582471f82785a87eada114150, data reload: false
------ Round 1 ----------------------------------
q1 6425 5155 5129 5129
q2 592 157 143 143
q3 1423 1145 1155 1145
q4 1120 812 815 812
q5 2932 2830 2886 2830
q6 221 137 137 137
q7 960 582 551 551
q8 2033 2033 2006 2006
q9 6509 6480 6383 6383
q10 7909 3041 3092 3041
q11 448 210 212 210
q12 395 241 240 240
q13 18050 3665 3636 3636
q14 249 216 225 216
q15 593 549 557 549
q16 448 417 422 417
q17 977 468 495 468
q18 7627 6702 6752 6702
q19 1612 1324 1343 1324
q20 683 339 356 339
q21 2754 2330 2417 2330
q22 389 321 326 321
Total cold run time: 64349 ms
Total hot run time: 38929 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5152 5035 5076 5035
q2 333 260 267 260
q3 3337 3280 3260 3260
q4 2089 2010 2031 2010
q5 5827 5843 5806 5806
q6 217 126 127 126
q7 2305 1915 1893 1893
q8 3413 3476 3472 3472
q9 8864 8744 8754 8744
q10 3817 3825 3871 3825
q11 597 506 474 474
q12 826 645 628 628
q13 6283 3259 3244 3244
q14 288 265 268 265
q15 600 542 546 542
q16 563 498 498 498
q17 1926 1765 1776 1765
q18 8723 8424 8291 8291
q19 1632 1638 1601 1601
q20 2206 1971 1970 1970
q21 5674 5348 5249 5249
q22 550 509 484 484
Total cold run time: 65222 ms
Total hot run time: 59442 ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](inverted index) strictly checkout inverted index properties [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29421:
URL: https://github.com/apache/doris/pull/29421#issuecomment-1873967805
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](inverted index) strictly checkout inverted index properties [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29421:
URL: https://github.com/apache/doris/pull/29421#issuecomment-1873990969
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.64 seconds
stream load tsv: 566 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.2 seconds inserted 10000000 Rows, about 354K ops/s
storage size: 17183722935 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org