You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "airborne12 (via GitHub)" <gi...@apache.org> on 2023/12/12 07:50:38 UTC
[PR] [Refact](inverted index) comment out block_column_predicate for inverted index and refact some code [doris]
airborne12 opened a new pull request, #28281:
URL: https://github.com/apache/doris/pull/28281
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refact](inverted index) comment out block_column_predicate for inverted index and refact some code [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28281:
URL: https://github.com/apache/doris/pull/28281#issuecomment-1851733043
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refact](inverted index) comment out block_column_predicate for inverted index and refact some code [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28281:
URL: https://github.com/apache/doris/pull/28281#issuecomment-1851615391
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit cf1a166f3268bb1b1dab3e43c1c1bb94e43661f7, data reload: true
run tpch-sf100 query with default conf and session variables
q1 4899 4447 4375 4375
q2 376 117 121 117
q3 1525 1234 1244 1234
q4 3416 853 883 853
q5 3381 3184 3198 3184
q6 236 122 125 122
q7 998 473 477 473
q8 3851 2138 2193 2138
q9 6805 6657 6707 6657
q10 3236 3269 3291 3269
q11 329 187 204 187
q12 363 207 206 206
q13 4565 3816 3833 3816
q14 240 206 213 206
q15 561 520 523 520
q16 437 378 381 378
q17 1001 622 561 561
q18 7239 7174 6976 6976
q19 1585 1457 1458 1457
q20 584 318 300 300
q21 3072 2689 2662 2662
q22 339 281 276 276
Total cold run time: 49038 ms
Total hot run time: 39967 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4406 4426 4392 4392
q2 265 160 170 160
q3 3496 3498 3486 3486
q4 2385 2367 2389 2367
q5 5729 5730 5728 5728
q6 232 122 119 119
q7 2393 1839 1861 1839
q8 3526 3518 3539 3518
q9 8995 8973 9001 8973
q10 3943 4012 4055 4012
q11 495 394 375 375
q12 766 605 588 588
q13 4300 3598 3543 3543
q14 292 258 246 246
q15 574 531 516 516
q16 496 448 425 425
q17 1875 1867 1840 1840
q18 8670 8191 8199 8191
q19 1723 1752 1748 1748
q20 2251 1941 1923 1923
q21 6498 6148 6155 6148
q22 503 407 421 407
Total cold run time: 63813 ms
Total hot run time: 60544 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refact](inverted index) comment out block_column_predicate for inverted index and refact some code [doris]
Posted by "airborne12 (via GitHub)" <gi...@apache.org>.
airborne12 commented on PR #28281:
URL: https://github.com/apache/doris/pull/28281#issuecomment-1851464993
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refact](inverted index) comment out block_column_predicate for inverted index and refact some code [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28281:
URL: https://github.com/apache/doris/pull/28281#issuecomment-1851539976
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.84 seconds
stream load tsv: 577 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 34 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 29.4 seconds inserted 10000000 Rows, about 340K ops/s
storage size: 17219951959 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refact](inverted index) comment out block_column_predicate for inverted index and refact some code [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28281:
URL: https://github.com/apache/doris/pull/28281#issuecomment-1851605411
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refact](inverted index) comment out block_column_predicate for inverted index and refact some code [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #28281:
URL: https://github.com/apache/doris/pull/28281#discussion_r1423582235
##########
be/src/olap/rowset/segment_v2/segment_iterator.cpp:
##########
@@ -1100,6 +1100,36 @@ bool SegmentIterator::_need_read_data(ColumnId cid) {
return true;
}
+bool SegmentIterator::_is_target_expr_match_predicate(const vectorized::VExprSPtr& expr,
Review Comment:
warning: method '_is_target_expr_match_predicate' can be made static [readability-convert-member-functions-to-static]
be/src/olap/rowset/segment_v2/segment_iterator.h:377:
```diff
- bool _is_target_expr_match_predicate(const vectorized::VExprSPtr& expr,
+ static bool _is_target_expr_match_predicate(const vectorized::VExprSPtr& expr,
```
##########
be/src/olap/rowset/segment_v2/segment_iterator.h:
##########
@@ -361,6 +362,22 @@ class SegmentIterator : public RowwiseIterator {
return 0;
}
+ bool _is_match_predicate_and_not_remaining(
Review Comment:
warning: method '_is_match_predicate_and_not_remaining' can be made static [readability-convert-member-functions-to-static]
```suggestion
static bool _is_match_predicate_and_not_remaining(
```
##########
be/src/olap/rowset/segment_v2/segment_iterator.h:
##########
@@ -361,6 +362,22 @@
return 0;
}
+ bool _is_match_predicate_and_not_remaining(
+ ColumnPredicate* pred, const std::vector<ColumnPredicate*>& remaining_predicates) {
+ return pred->type() == PredicateType::MATCH &&
+ std::find(remaining_predicates.begin(), remaining_predicates.end(), pred) ==
+ remaining_predicates.end();
+ }
+
+ void _delete_expr_from_conjunct_roots(const vectorized::VExprSPtr& expr,
Review Comment:
warning: method '_delete_expr_from_conjunct_roots' can be made static [readability-convert-member-functions-to-static]
```suggestion
static void _delete_expr_from_conjunct_roots(const vectorized::VExprSPtr& expr,
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Refact](inverted index) comment out block_column_predicate for inverted index and refact some code [doris]
Posted by "qidaye (via GitHub)" <gi...@apache.org>.
qidaye merged PR #28281:
URL: https://github.com/apache/doris/pull/28281
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org