You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "airborne12 (via GitHub)" <gi...@apache.org> on 2023/12/07 03:33:02 UTC
[PR] [Fix](inverted index) fix need read data optimize problem [doris]
airborne12 opened a new pull request, #28104:
URL: https://github.com/apache/doris/pull/28104
## Proposed changes
Issue Number: close #xxx
This fix addresses an issue encountered when querying an inverted index table with a query like: SELECT COUNT() FROM table WHERE index_column > 1 AND index_column < 2.
In such scenarios, if the predicate index_column > 1 inadvertently skips using the index, but index_column < 2 utilizes the index for querying, the index_column gets incorrectly marked as not needing data read. This behavior leads to incorrect query results. The fix ensures accurate marking of columns for data reads in these cases.
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](inverted index) fix need read data optimize problem [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28104:
URL: https://github.com/apache/doris/pull/28104#issuecomment-1844656251
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](inverted index) fix need read data optimize problem [doris]
Posted by "eldenmoon (via GitHub)" <gi...@apache.org>.
eldenmoon merged PR #28104:
URL: https://github.com/apache/doris/pull/28104
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](inverted index) fix need read data optimize problem [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28104:
URL: https://github.com/apache/doris/pull/28104#issuecomment-1844318771
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.95 seconds
stream load tsv: 584 seconds loaded 74807831229 Bytes, about 122 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17194922485 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](inverted index) fix need read data optimize problem [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28104:
URL: https://github.com/apache/doris/pull/28104#issuecomment-1844655852
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](inverted index) fix need read data optimize problem [doris]
Posted by "airborne12 (via GitHub)" <gi...@apache.org>.
airborne12 commented on PR #28104:
URL: https://github.com/apache/doris/pull/28104#issuecomment-1844199928
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](inverted index) fix need read data optimize problem [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28104:
URL: https://github.com/apache/doris/pull/28104#issuecomment-1844205489
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](inverted index) fix need read data optimize problem [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28104:
URL: https://github.com/apache/doris/pull/28104#issuecomment-1844316281
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit d69061dedb4adab39d97f8e020743fe706377912, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4736 4518 4480 4480
q2 361 158 158 158
q3 1462 1206 1264 1206
q4 1116 924 942 924
q5 3190 3171 3163 3163
q6 246 130 131 130
q7 974 497 492 492
q8 2200 2226 2181 2181
q9 6656 6686 6626 6626
q10 3194 3262 3262 3262
q11 315 206 210 206
q12 374 215 214 214
q13 4542 3815 3780 3780
q14 245 220 219 219
q15 566 514 521 514
q16 452 386 405 386
q17 998 582 572 572
q18 7641 7150 6922 6922
q19 1501 1376 1468 1376
q20 587 326 335 326
q21 3154 2676 2760 2676
q22 358 291 293 291
Total cold run time: 44868 ms
Total hot run time: 40104 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4424 4367 4378 4367
q2 271 162 173 162
q3 3531 3524 3536 3524
q4 2383 2375 2382 2375
q5 5727 5724 5738 5724
q6 240 121 123 121
q7 2362 1863 1854 1854
q8 3512 3493 3513 3493
q9 9075 9012 9022 9012
q10 3903 3994 3985 3985
q11 503 389 374 374
q12 764 599 593 593
q13 4303 3581 3572 3572
q14 279 270 266 266
q15 574 521 507 507
q16 515 448 461 448
q17 1861 1866 1837 1837
q18 8688 8316 8335 8316
q19 1734 1777 1750 1750
q20 2253 1957 1940 1940
q21 6577 6156 6153 6153
q22 500 412 408 408
Total cold run time: 63979 ms
Total hot run time: 60781 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org