You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2019/12/24 04:22:03 UTC
[GitHub] [incubator-doris] gaodayue opened a new issue #2545: [Segment V2]
lazy materialization read
gaodayue opened a new issue #2545: [Segment V2] lazy materialization read
URL: https://github.com/apache/incubator-doris/issues/2545
Part of #1652
Current read path of SegmentIterator
----
1. apply short key index and various column indexes to get the row ranges (ordinals of rows) to scan
2. read all return columns according to row ranges
3. evaluate column predicates on the RowBlockV2 to further prune rows
Problem
----
When the column predicates at step 3 filter a large proportion of rows in RowBlockV2, most values of non-predicate columns we read at step 2 are thrown, i.e lots of useless work.
Lazy materialization read
----
This is an optimization suitable for column stores and can be used to solve the above problem.
With lazy materialization, the read path changes to
1. apply short key index and various column indexes to get the row ranges (ordinals of rows) to scan (unchanged)
2. read *only predicate columns* according to row ranges
3. evaluate column predicates on the RowBlockV2 to further prune rows, the selected rows are set into a selection vector
4. read the remaining columns based on the *selection vector* of RowBlockV2
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] imay closed issue #2545: [Segment V2] lazy
materialization read
Posted by GitBox <gi...@apache.org>.
imay closed issue #2545: [Segment V2] lazy materialization read
URL: https://github.com/apache/incubator-doris/issues/2545
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org