You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "HappenLee (via GitHub)" <gi...@apache.org> on 2024/01/11 03:39:26 UTC
[PR] [Session](rf) Change the default min size of bf runtime filter [doris]
HappenLee opened a new pull request, #29837:
URL: https://github.com/apache/doris/pull/29837
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Session](rf) Change the default min size of bf runtime filter to 2048 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29837:
URL: https://github.com/apache/doris/pull/29837#issuecomment-1886397255
<details>
<summary>TPC-H: <b>Total hot run time: 38518 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d8af6ea6fb8fed31ac872cf84d207da2fc729b8a, data reload: false
------ Round 1 ----------------------------------
q1 18019 5017 4923 4923
q2 2037 136 129 129
q3 10691 1141 1192 1141
q4 10256 754 810 754
q5 7775 3161 3151 3151
q6 198 123 123 123
q7 860 514 489 489
q8 9281 1949 2003 1949
q9 7327 6417 6440 6417
q10 8250 3095 3107 3095
q11 411 213 212 212
q12 364 189 192 189
q13 18101 3417 3401 3401
q14 240 216 208 208
q15 555 518 510 510
q16 433 377 387 377
q17 932 483 563 483
q18 7245 6673 6626 6626
q19 1584 1361 1354 1354
q20 586 305 301 301
q21 2772 2395 2414 2395
q22 357 303 291 291
Total cold run time: 108274 ms
Total hot run time: 38518 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4968 4968 4950 4950
q2 308 202 195 195
q3 3316 3300 3285 3285
q4 2239 2255 2241 2241
q5 5785 5754 5774 5754
q6 195 116 117 116
q7 2307 1834 1840 1834
q8 3445 3517 3571 3517
q9 8823 8806 8783 8783
q10 3738 3821 3819 3819
q11 550 438 426 426
q12 778 596 603 596
q13 7280 3216 3192 3192
q14 277 266 273 266
q15 558 518 497 497
q16 514 470 475 470
q17 2049 2047 2058 2047
q18 8813 8368 8262 8262
q19 1590 1622 1616 1616
q20 2144 1923 1946 1923
q21 6019 5517 5767 5517
q22 565 444 493 444
Total cold run time: 66261 ms
Total hot run time: 59750 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Session](rf) Change the default min size of bf runtime filter to 2048 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29837:
URL: https://github.com/apache/doris/pull/29837#issuecomment-1886422758
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit d8af6ea6fb8fed31ac872cf84d207da2fc729b8a with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 12.9 seconds inserted 10000000 Rows, about 775K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Session](rf) Change the default min size of bf runtime filter to 2048 [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29837:
URL: https://github.com/apache/doris/pull/29837#issuecomment-1886192311
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Session](rf) Change the default min size of bf runtime filter to 2048 [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29837:
URL: https://github.com/apache/doris/pull/29837#issuecomment-1886192566
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Session](rf) Change the default min size of bf runtime filter to 2048 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29837:
URL: https://github.com/apache/doris/pull/29837#issuecomment-1886419261
<details>
<summary>ClickBench: <b>Total hot run time: 30.89 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d8af6ea6fb8fed31ac872cf84d207da2fc729b8a, data reload: false
query1 0.07 0.05 0.05
query2 0.06 0.03 0.02
query3 0.25 0.11 0.12
query4 1.76 0.13 0.11
query5 0.52 0.52 0.52
query6 1.39 0.64 0.62
query7 0.01 0.01 0.02
query8 0.04 0.02 0.02
query9 0.55 0.50 0.50
query10 0.55 0.55 0.55
query11 0.13 0.10 0.09
query12 0.11 0.10 0.09
query13 0.61 0.60 0.60
query14 0.78 0.78 0.82
query15 0.83 0.80 0.80
query16 0.36 0.37 0.34
query17 0.98 0.97 0.97
query18 0.24 0.26 0.23
query19 1.80 1.81 1.81
query20 0.01 0.00 0.00
query21 15.40 0.58 0.58
query22 2.45 2.58 2.26
query23 17.76 0.78 0.79
query24 16.37 0.59 0.58
query25 2.18 0.15 0.14
query26 0.14 0.14 0.13
query27 0.15 0.15 0.15
query28 6.69 0.84 0.81
query29 12.54 3.14 3.16
query30 0.59 0.47 0.47
query31 2.78 0.35 0.36
query32 3.39 0.48 0.49
query33 3.21 3.20 3.20
query34 15.86 4.16 4.17
query35 4.15 4.16 4.16
query36 1.10 1.04 1.03
query37 0.07 0.05 0.05
query38 0.03 0.02 0.02
query39 0.02 0.02 0.02
query40 0.16 0.12 0.13
query41 0.08 0.01 0.01
query42 0.02 0.02 0.01
query43 0.02 0.02 0.02
Total cold run time: 116.21 s
Total hot run time: 30.89 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Session](rf) Change the default min size of bf runtime filter to 2048 [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee merged PR #29837:
URL: https://github.com/apache/doris/pull/29837
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Session](rf) Change the default min size of bf runtime filter to 2048 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29837:
URL: https://github.com/apache/doris/pull/29837#issuecomment-1886406151
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.55 seconds
stream load tsv: 558 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.1 seconds inserted 10000000 Rows, about 473K ops/s
storage size: 17183685903 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Session](rf) Change the default min size of bf runtime filter to 2048 [doris]
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on PR #29837:
URL: https://github.com/apache/doris/pull/29837#issuecomment-1886345688
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Session](rf) Change the default min size of bf runtime filter to 2048 [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29837:
URL: https://github.com/apache/doris/pull/29837#issuecomment-1886408998
<details>
<summary>TPC-DS: <b>Total hot run time: 179567 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d8af6ea6fb8fed31ac872cf84d207da2fc729b8a, data reload: false
query1 938 338 333 333
query2 6717 1889 1970 1889
query3 6712 213 205 205
query4 27312 22266 22176 22176
query5 4733 499 500 499
query6 263 184 192 184
query7 4615 286 276 276
query8 221 198 205 198
query9 9195 2706 2718 2706
query10 661 246 236 236
query11 16299 15415 15416 15415
query12 126 74 76 74
query13 1705 385 413 385
query14 11505 7358 7175 7175
query15 235 198 194 194
query16 6553 251 243 243
query17 1912 484 479 479
query18 1954 275 259 259
query19 289 151 145 145
query20 84 75 77 75
query21 200 135 126 126
query22 4886 4741 4820 4741
query23 31856 30935 30969 30935
query24 12221 2842 2813 2813
query25 552 331 335 331
query26 1749 157 157 157
query27 2955 278 285 278
query28 7048 1859 1866 1859
query29 2072 711 668 668
query30 282 135 141 135
query31 1008 765 784 765
query32 89 67 59 59
query33 714 257 254 254
query34 1074 467 480 467
query35 929 744 746 744
query36 1425 1241 1233 1233
query37 184 68 68 68
query38 3398 3318 3278 3278
query39 1338 1299 1279 1279
query40 339 101 89 89
query41 41 35 35 35
query42 102 98 97 97
query43 543 484 504 484
query44 1094 713 727 713
query45 219 183 185 183
query46 1062 682 680 680
query47 1547 1556 1592 1556
query48 392 333 322 322
query49 1283 316 317 316
query50 707 320 319 319
query51 5286 5261 5213 5213
query52 101 89 93 89
query53 361 278 299 278
query54 2087 1598 1622 1598
query55 97 90 90 90
query56 222 200 199 199
query57 1015 935 969 935
query58 218 195 200 195
query59 2732 2530 2471 2471
query60 239 214 212 212
query61 83 81 82 81
query62 596 392 406 392
query63 303 292 295 292
query64 5849 1656 1654 1654
query65 3361 3267 3282 3267
query66 1393 340 332 332
query67 15471 14990 15059 14990
query68 12606 550 537 537
query69 659 386 384 384
query70 1707 1545 1594 1545
query71 500 235 248 235
query72 4924 2814 2859 2814
query73 2990 324 319 319
query74 7086 6466 6555 6466
query75 5316 2325 2272 2272
query76 6376 866 1120 866
query77 668 275 284 275
query78 9914 8538 8700 8538
query79 1047 527 504 504
query80 530 351 354 351
query81 452 207 211 207
query82 207 95 86 86
query83 178 120 123 120
query84 276 73 76 73
query85 1061 338 337 337
query86 400 390 385 385
query87 3591 3420 3362 3362
query88 3239 2248 2245 2245
query89 470 378 398 378
query90 1900 209 201 201
query91 157 125 127 125
query92 64 58 54 54
query93 1386 439 427 427
query94 1296 185 188 185
query95 519 490 472 472
query96 619 324 326 324
query97 4283 4132 4175 4132
query98 210 190 188 188
query99 1000 742 754 742
Total cold run time: 298383 ms
Total hot run time: 179567 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org