You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Gabriel39 (via GitHub)" <gi...@apache.org> on 2024/01/11 08:22:40 UTC
[PR] [pipelineX](fix) Fix incorrect runtime filter [doris]
Gabriel39 opened a new pull request, #29860:
URL: https://github.com/apache/doris/pull/29860
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886666964
<details>
<summary>TPC-DS: <b>Total hot run time: 179537 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f729b362cfb648a1d73f79631b35751b264abb34, data reload: false
query1 941 333 323 323
query2 6712 1922 1827 1827
query3 6699 212 210 210
query4 25894 22247 22123 22123
query5 5163 520 541 520
query6 272 194 185 185
query7 4624 284 281 281
query8 247 208 200 200
query9 9224 2754 2925 2754
query10 457 250 241 241
query11 16151 15376 15326 15326
query12 140 78 72 72
query13 1703 399 390 390
query14 11737 7355 7288 7288
query15 238 191 198 191
query16 6479 261 253 253
query17 1879 481 473 473
query18 1943 266 278 266
query19 279 153 156 153
query20 84 79 76 76
query21 185 137 126 126
query22 5061 4719 4714 4714
query23 32226 30935 30811 30811
query24 10957 2890 2881 2881
query25 559 337 329 329
query26 1346 150 160 150
query27 2826 281 281 281
query28 6949 1889 1872 1872
query29 1160 678 655 655
query30 273 138 139 138
query31 980 777 788 777
query32 98 63 59 59
query33 704 247 263 247
query34 972 484 468 468
query35 910 765 780 765
query36 1395 1222 1244 1222
query37 98 78 73 73
query38 3412 3282 3252 3252
query39 1346 1280 1267 1267
query40 215 100 100 100
query41 37 36 36 36
query42 107 98 99 98
query43 534 517 517 517
query44 1095 715 722 715
query45 212 188 182 182
query46 1061 665 676 665
query47 1634 1636 1604 1604
query48 403 323 340 323
query49 1151 334 323 323
query50 706 317 328 317
query51 5299 5211 5233 5211
query52 102 90 94 90
query53 366 300 293 293
query54 2091 1645 1626 1626
query55 106 85 90 85
query56 215 201 203 201
query57 1033 965 946 946
query58 216 201 207 201
query59 2754 2563 2461 2461
query60 233 218 232 218
query61 86 81 80 80
query62 605 419 402 402
query63 299 297 294 294
query64 4944 1676 1742 1676
query65 3357 3258 3285 3258
query66 1049 339 318 318
query67 15604 15281 15051 15051
query68 13021 553 525 525
query69 670 375 392 375
query70 1685 1499 1529 1499
query71 506 247 243 243
query72 5091 2858 2814 2814
query73 3046 311 317 311
query74 6923 6406 6490 6406
query75 5376 2364 2345 2345
query76 6380 900 1141 900
query77 697 278 266 266
query78 9993 8667 8465 8465
query79 1723 525 504 504
query80 579 345 345 345
query81 472 207 213 207
query82 215 92 92 92
query83 177 127 127 127
query84 284 73 74 73
query85 893 345 333 333
query86 378 378 402 378
query87 3598 3411 3387 3387
query88 3247 2229 2253 2229
query89 473 401 388 388
query90 1941 216 207 207
query91 165 128 127 127
query92 59 55 56 55
query93 1321 425 422 422
query94 1300 183 186 183
query95 526 498 463 463
query96 615 318 317 317
query97 4301 4194 4165 4165
query98 208 198 188 188
query99 1001 748 668 668
Total cold run time: 294701 ms
Total hot run time: 179537 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886964957
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886611651
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886964813
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886602499
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886656103
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.2 seconds
stream load tsv: 573 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 21.6 seconds inserted 10000000 Rows, about 462K ops/s
storage size: 17184234905 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886632903
TeamCity be ut coverage result:
Function Coverage: 36.72% (8650/23558)
Line Coverage: 28.75% (70468/245121)
Region Coverage: 27.67% (36414/131584)
Branch Coverage: 24.39% (18639/76418)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f729b362cfb648a1d73f79631b35751b264abb34_f729b362cfb648a1d73f79631b35751b264abb34/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886648066
<details>
<summary>TPC-H: <b>Total hot run time: 38821 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f729b362cfb648a1d73f79631b35751b264abb34, data reload: false
------ Round 1 ----------------------------------
q1 17968 5069 4991 4991
q2 2047 142 141 141
q3 10678 1123 1192 1123
q4 10240 797 848 797
q5 7783 3131 3141 3131
q6 205 129 128 128
q7 873 517 495 495
q8 9238 1932 2001 1932
q9 7353 6518 6474 6474
q10 8247 3097 3067 3067
q11 420 202 218 202
q12 367 193 201 193
q13 18123 3463 3440 3440
q14 237 221 211 211
q15 559 517 513 513
q16 430 376 375 375
q17 940 559 527 527
q18 7332 6737 6687 6687
q19 1592 1389 1404 1389
q20 586 319 306 306
q21 2788 2401 2410 2401
q22 347 315 298 298
Total cold run time: 108353 ms
Total hot run time: 38821 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5007 5056 5050 5050
q2 312 195 195 195
q3 3343 3316 3321 3316
q4 2256 2280 2260 2260
q5 5825 5793 5810 5793
q6 199 120 123 120
q7 2314 1884 1863 1863
q8 3448 3568 3586 3568
q9 8900 8855 8775 8775
q10 3761 3836 3836 3836
q11 544 436 427 427
q12 787 607 616 607
q13 6448 3225 3200 3200
q14 285 262 260 260
q15 579 510 506 506
q16 501 468 470 468
q17 2051 2005 2020 2005
q18 8778 8303 8277 8277
q19 1626 1634 1633 1633
q20 2167 1932 1937 1932
q21 6140 5688 5723 5688
q22 549 469 458 458
Total cold run time: 65820 ms
Total hot run time: 60237 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886682231
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit f729b362cfb648a1d73f79631b35751b264abb34 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 13.6 seconds inserted 10000000 Rows, about 735K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886676269
<details>
<summary>ClickBench: <b>Total hot run time: 30.28 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f729b362cfb648a1d73f79631b35751b264abb34, data reload: false
query1 0.06 0.06 0.05
query2 0.06 0.02 0.02
query3 0.25 0.12 0.10
query4 1.78 0.12 0.12
query5 0.52 0.52 0.52
query6 1.38 0.64 0.65
query7 0.02 0.01 0.01
query8 0.03 0.03 0.02
query9 0.56 0.51 0.49
query10 0.55 0.55 0.56
query11 0.12 0.09 0.09
query12 0.11 0.09 0.09
query13 0.61 0.59 0.60
query14 0.78 0.80 0.77
query15 0.83 0.80 0.79
query16 0.34 0.35 0.35
query17 0.97 1.00 1.02
query18 0.26 0.26 0.26
query19 1.86 1.79 1.78
query20 0.01 0.01 0.01
query21 15.43 0.58 0.60
query22 2.81 2.10 1.60
query23 17.16 0.77 0.72
query24 16.05 0.64 0.59
query25 2.16 0.15 0.15
query26 0.14 0.13 0.13
query27 0.14 0.15 0.14
query28 6.33 0.84 0.79
query29 12.55 3.20 3.17
query30 0.52 0.50 0.49
query31 2.78 0.34 0.36
query32 3.38 0.48 0.49
query33 3.24 3.21 3.20
query34 15.92 4.19 4.18
query35 4.18 4.17 4.17
query36 1.11 1.05 1.05
query37 0.06 0.04 0.04
query38 0.04 0.03 0.03
query39 0.03 0.01 0.01
query40 0.17 0.13 0.14
query41 0.08 0.02 0.01
query42 0.02 0.02 0.01
query43 0.03 0.02 0.02
Total cold run time: 115.43 s
Total hot run time: 30.28 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 merged PR #29860:
URL: https://github.com/apache/doris/pull/29860
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886601334
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](fix) Fix incorrect runtime filter [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29860:
URL: https://github.com/apache/doris/pull/29860#issuecomment-1886612407
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org