You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "airborne12 (via GitHub)" <gi...@apache.org> on 2024/07/11 06:03:25 UTC
[PR] [Update](inverted index) add column name contains in debug point for no need read data opt [doris]
airborne12 opened a new pull request, #37649:
URL: https://github.com/apache/doris/pull/37649
## Proposed changes
We need to verify the false negative situation when applying the "no need to read data" optimization in inverted index predicates. In this PR, we add the column name to the debug point parameters, allowing us to check if the optimization is functioning correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Update](inverted index) Add column name to debug point for "no need to read data" optimization [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37649:
URL: https://github.com/apache/doris/pull/37649#issuecomment-2224275010
<details>
<summary>TPC-H: <b>Total hot run time: 39941 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c9216cb66ec92c480563b0474a8797ccef220760, data reload: false
------ Round 1 ----------------------------------
q1 17874 4451 4328 4328
q2 2596 195 187 187
q3 11163 1231 1166 1166
q4 10724 809 784 784
q5 7629 2693 2778 2693
q6 226 138 144 138
q7 960 608 621 608
q8 9575 2300 2083 2083
q9 8771 6534 6516 6516
q10 8729 3708 3747 3708
q11 449 238 234 234
q12 398 241 229 229
q13 18719 2964 2981 2964
q14 283 230 241 230
q15 514 477 499 477
q16 502 383 374 374
q17 963 650 648 648
q18 7958 7427 7390 7390
q19 4265 1549 1459 1459
q20 685 325 325 325
q21 4969 3066 3206 3066
q22 385 334 334 334
Total cold run time: 118337 ms
Total hot run time: 39941 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4400 4236 4236 4236
q2 372 269 274 269
q3 2960 2749 2753 2749
q4 1876 1576 1579 1576
q5 5281 5279 5311 5279
q6 218 133 133 133
q7 2144 1702 1737 1702
q8 3176 3325 3280 3280
q9 8380 8378 8401 8378
q10 3905 3708 3650 3650
q11 578 487 485 485
q12 774 617 598 598
q13 16322 2995 3000 2995
q14 305 268 264 264
q15 516 476 476 476
q16 495 428 432 428
q17 1781 1471 1469 1469
q18 7679 7479 7361 7361
q19 1658 1660 1574 1574
q20 2027 1798 1782 1782
q21 4834 4616 4625 4616
q22 594 519 518 518
Total cold run time: 70275 ms
Total hot run time: 53818 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Update](inverted index) Add column name to debug point for "no need to read data" optimization [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37649:
URL: https://github.com/apache/doris/pull/37649#issuecomment-2222110145
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Update](inverted index) Add column name to debug point for "no need to read data" optimization [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37649:
URL: https://github.com/apache/doris/pull/37649#issuecomment-2224292567
<details>
<summary>TPC-DS: <b>Total hot run time: 174461 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c9216cb66ec92c480563b0474a8797ccef220760, data reload: false
query1 913 368 360 360
query2 6458 2412 2439 2412
query3 6646 215 228 215
query4 23792 17633 17221 17221
query5 4249 489 476 476
query6 282 182 195 182
query7 4595 296 290 290
query8 319 317 292 292
query9 8672 2480 2473 2473
query10 460 282 273 273
query11 10712 10088 10049 10049
query12 132 88 82 82
query13 1687 372 357 357
query14 10199 7676 7722 7676
query15 233 189 187 187
query16 7865 318 309 309
query17 1808 549 521 521
query18 1979 274 266 266
query19 211 149 150 149
query20 106 81 80 80
query21 204 126 128 126
query22 4289 4004 3973 3973
query23 33733 32995 32950 32950
query24 11872 2998 2900 2900
query25 672 367 383 367
query26 1769 153 151 151
query27 2960 274 275 274
query28 7745 2123 2127 2123
query29 1131 649 628 628
query30 279 180 145 145
query31 996 728 737 728
query32 105 54 55 54
query33 769 295 291 291
query34 942 493 487 487
query35 686 611 569 569
query36 1107 919 909 909
query37 180 83 79 79
query38 2878 2755 2751 2751
query39 870 802 800 800
query40 271 118 119 118
query41 54 52 52 52
query42 122 102 98 98
query43 575 555 590 555
query44 1199 752 730 730
query45 191 167 164 164
query46 1091 725 731 725
query47 1849 1751 1749 1749
query48 380 312 308 308
query49 1177 416 407 407
query50 796 399 398 398
query51 6902 6758 6748 6748
query52 108 94 98 94
query53 392 296 296 296
query54 994 462 447 447
query55 78 73 77 73
query56 284 266 279 266
query57 1130 1055 1049 1049
query58 254 239 240 239
query59 3416 3364 3365 3364
query60 325 273 283 273
query61 110 117 104 104
query62 816 667 652 652
query63 317 291 286 286
query64 10490 2208 1671 1671
query65 3171 3102 3132 3102
query66 1380 345 339 339
query67 15364 14788 14848 14788
query68 4568 553 571 553
query69 480 343 335 335
query70 1204 1145 1122 1122
query71 403 287 286 286
query72 7209 5645 5532 5532
query73 764 332 337 332
query74 6065 5503 5453 5453
query75 3460 2694 2711 2694
query76 2833 979 930 930
query77 440 310 311 310
query78 9592 9101 9090 9090
query79 2272 528 535 528
query80 1332 540 499 499
query81 580 223 222 222
query82 784 142 140 140
query83 240 180 175 175
query84 247 95 92 92
query85 1958 368 364 364
query86 489 328 308 308
query87 3229 3125 3138 3125
query88 4306 2512 2487 2487
query89 487 390 380 380
query90 1849 200 198 198
query91 146 118 116 116
query92 70 52 55 52
query93 2367 513 514 513
query94 1274 223 227 223
query95 416 333 338 333
query96 598 283 281 281
query97 3191 3047 3040 3040
query98 222 198 200 198
query99 1624 1280 1284 1280
Total cold run time: 282440 ms
Total hot run time: 174461 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Update](inverted index) Add column name to debug point for "no need to read data" optimization [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37649:
URL: https://github.com/apache/doris/pull/37649#issuecomment-2222110685
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Update](inverted index) Add column name to debug point for "no need to read data" optimization [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37649:
URL: https://github.com/apache/doris/pull/37649#issuecomment-2224297070
<details>
<summary>ClickBench: <b>Total hot run time: 31.41 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c9216cb66ec92c480563b0474a8797ccef220760, data reload: false
query1 0.04 0.03 0.03
query2 0.08 0.04 0.04
query3 0.23 0.05 0.05
query4 1.67 0.06 0.07
query5 0.50 0.49 0.50
query6 1.13 0.73 0.73
query7 0.02 0.01 0.01
query8 0.06 0.05 0.04
query9 0.55 0.48 0.49
query10 0.54 0.55 0.54
query11 0.15 0.12 0.12
query12 0.14 0.12 0.12
query13 0.59 0.59 0.58
query14 0.77 0.77 0.78
query15 0.86 0.81 0.81
query16 0.36 0.37 0.37
query17 1.02 1.02 1.03
query18 0.22 0.21 0.22
query19 1.79 1.83 1.68
query20 0.01 0.00 0.01
query21 15.42 0.73 0.66
query22 4.18 6.21 2.71
query23 18.29 1.33 1.34
query24 2.11 0.22 0.23
query25 0.16 0.08 0.09
query26 0.29 0.21 0.21
query27 0.46 0.23 0.23
query28 13.25 1.02 0.99
query29 12.58 3.30 3.24
query30 0.25 0.06 0.05
query31 2.89 0.40 0.39
query32 3.27 0.47 0.46
query33 2.90 2.91 2.90
query34 17.03 4.32 4.40
query35 4.44 4.52 4.42
query36 0.66 0.46 0.49
query37 0.19 0.16 0.16
query38 0.15 0.15 0.15
query39 0.05 0.04 0.04
query40 0.16 0.12 0.12
query41 0.10 0.04 0.05
query42 0.06 0.04 0.06
query43 0.04 0.04 0.04
Total cold run time: 109.66 s
Total hot run time: 31.41 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Update](inverted index) add column name contains in debug point for no need read data opt [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37649:
URL: https://github.com/apache/doris/pull/37649#issuecomment-2222103304
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Update](inverted index) Add column name to debug point for "no need to read data" optimization [doris]
Posted by "airborne12 (via GitHub)" <gi...@apache.org>.
airborne12 merged PR #37649:
URL: https://github.com/apache/doris/pull/37649
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Update](inverted index) Add column name to debug point for "no need to read data" optimization [doris]
Posted by "airborne12 (via GitHub)" <gi...@apache.org>.
airborne12 commented on PR #37649:
URL: https://github.com/apache/doris/pull/37649#issuecomment-2224260676
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Update](inverted index) Add column name to debug point for "no need to read data" optimization [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37649:
URL: https://github.com/apache/doris/pull/37649#issuecomment-2222110645
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org