You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Tanya-W (via GitHub)" <gi...@apache.org> on 2023/06/08 10:50:56 UTC
[GitHub] [doris] Tanya-W opened a new pull request, #20605: [enhancement](index) Nereids support no need to read raw data for index column that only in filter conditions
Tanya-W opened a new pull request, #20605:
URL: https://github.com/apache/doris/pull/20605
## Proposed changes
In nereids, support the optimization for no need read raw data of index column when the index column only in the filter conditions.
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morrySnow commented on a diff in pull request #20605: [enhancement](index) Nereids support no need to read raw data for index column that only in filter conditions
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on code in PR #20605:
URL: https://github.com/apache/doris/pull/20605#discussion_r1222870206
##########
fe/fe-core/src/main/java/org/apache/doris/planner/OriginalPlanner.java:
##########
@@ -572,50 +572,14 @@ private void pushSortToOlapScan() {
* column unique id for `A` and `B` will put into outputColumnUniqueIds.
*
*/
+ // this opt will only work with nereidsPlanner
private void pushOutColumnUniqueIdsToOlapScan(PlanFragment rootFragment, Analyzer analyzer) {
Set<Integer> outputColumnUniqueIds = new HashSet<>();
- ArrayList<Expr> outputExprs = rootFragment.getOutputExprs();
- for (Expr expr : outputExprs) {
- if (expr instanceof SlotRef) {
- if (((SlotRef) expr).getColumn() != null) {
- outputColumnUniqueIds.add(((SlotRef) expr).getColumn().getUniqueId());
- }
- }
- }
+ // add '-1', make compatible for nereidsPlanner
Review Comment:
change comment to "add '-1' to avoid the optimization of xxxxxxx, because yyyyy"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] airborne12 commented on a diff in pull request #20605: [enhancement](index) Nereids support no need to read raw data for index column that only in filter conditions
Posted by "airborne12 (via GitHub)" <gi...@apache.org>.
airborne12 commented on code in PR #20605:
URL: https://github.com/apache/doris/pull/20605#discussion_r1222893490
##########
be/src/olap/rowset/segment_v2/segment_iterator.cpp:
##########
@@ -250,6 +250,10 @@ Status SegmentIterator::init(const StorageReadOptions& opts) {
if (_char_type_idx.empty() && _char_type_idx_no_0.empty()) {
_vec_init_char_column_id();
}
+
+ if (opts.output_columns != nullptr) {
+ _output_columns = *(opts.output_columns);
Review Comment:
Where is opts.output_columns from?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #20605: [enhancement](index) Nereids support no need to read raw data for index column that only in filter conditions
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #20605:
URL: https://github.com/apache/doris/pull/20605#issuecomment-1584616075
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #20605: [enhancement](index) Nereids support no need to read raw data for index column that only in filter conditions
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #20605:
URL: https://github.com/apache/doris/pull/20605#issuecomment-1582366555
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Tanya-W commented on a diff in pull request #20605: [enhancement](index) Nereids support no need to read raw data for index column that only in filter conditions
Posted by "Tanya-W (via GitHub)" <gi...@apache.org>.
Tanya-W commented on code in PR #20605:
URL: https://github.com/apache/doris/pull/20605#discussion_r1223746831
##########
be/src/olap/rowset/segment_v2/segment_iterator.cpp:
##########
@@ -250,6 +250,10 @@ Status SegmentIterator::init(const StorageReadOptions& opts) {
if (_char_type_idx.empty() && _char_type_idx_no_0.empty()) {
_vec_init_char_column_id();
}
+
+ if (opts.output_columns != nullptr) {
+ _output_columns = *(opts.output_columns);
Review Comment:
`_maybe_read_column_ids` in `NewOlapScanNode` -> `output_columns` in `ReaderParams` -> `output_columns` in `RowsetReaderContext` -> output_columns in StorageReadOptions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #20605: [enhancement](index) Nereids support no need to read raw data for index column that only in filter conditions
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #20605:
URL: https://github.com/apache/doris/pull/20605#issuecomment-1584105977
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #20605: [enhancement](index) Nereids support no need to read raw data for index column that only in filter conditions
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #20605:
URL: https://github.com/apache/doris/pull/20605#issuecomment-1583835451
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Tanya-W commented on pull request #20605: [enhancement](index) Nereids support no need to read raw data for index column that only in filter conditions
Posted by "Tanya-W (via GitHub)" <gi...@apache.org>.
Tanya-W commented on PR #20605:
URL: https://github.com/apache/doris/pull/20605#issuecomment-1584102110
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Tanya-W commented on pull request #20605: [enhancement](index) Nereids support no need to read raw data for index column that only in filter conditions
Posted by "Tanya-W (via GitHub)" <gi...@apache.org>.
Tanya-W commented on PR #20605:
URL: https://github.com/apache/doris/pull/20605#issuecomment-1583833729
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #20605: [enhancement](index) Nereids support no need to read raw data for index column that only in filter conditions
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #20605:
URL: https://github.com/apache/doris/pull/20605#issuecomment-1584616143
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xiaokang merged pull request #20605: [enhancement](index) Nereids support no need to read raw data for index column that only in filter conditions
Posted by "xiaokang (via GitHub)" <gi...@apache.org>.
xiaokang merged PR #20605:
URL: https://github.com/apache/doris/pull/20605
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Tanya-W commented on a diff in pull request #20605: [enhancement](index) Nereids support no need to read raw data for index column that only in filter conditions
Posted by "Tanya-W (via GitHub)" <gi...@apache.org>.
Tanya-W commented on code in PR #20605:
URL: https://github.com/apache/doris/pull/20605#discussion_r1223754422
##########
fe/fe-core/src/main/java/org/apache/doris/planner/OriginalPlanner.java:
##########
@@ -572,50 +572,14 @@ private void pushSortToOlapScan() {
* column unique id for `A` and `B` will put into outputColumnUniqueIds.
*
*/
+ // this opt will only work with nereidsPlanner
private void pushOutColumnUniqueIdsToOlapScan(PlanFragment rootFragment, Analyzer analyzer) {
Set<Integer> outputColumnUniqueIds = new HashSet<>();
- ArrayList<Expr> outputExprs = rootFragment.getOutputExprs();
- for (Expr expr : outputExprs) {
- if (expr instanceof SlotRef) {
- if (((SlotRef) expr).getColumn() != null) {
- outputColumnUniqueIds.add(((SlotRef) expr).getColumn().getUniqueId());
- }
- }
- }
+ // add '-1', make compatible for nereidsPlanner
Review Comment:
updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org