You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/02/28 08:53:59 UTC
[GitHub] [incubator-doris] EmmyMiao87 opened a new pull request #3018: Fix
the pre aggregation is difference between old and new selector
EmmyMiao87 opened a new pull request #3018: Fix the pre aggregation is difference between old and new selector
URL: https://github.com/apache/incubator-doris/pull/3018
The issue is #3016.
If there is no aggregated column in aggregate index, the index will be deduplicate table.
For example:
aggregate table (k1, k2, v1 sum)
mv index (k1, k2)
This kind of index is SPJG which same as select k1, k2 from aggregate_table group by k1, k2.
It also need to check the grouping column using following steps.
If there is no aggregated column in duplicate index, the index will be SPJ which passes the grouping verification directly.
Also after the supplement of index, the new candidate index should be checked the output columns also.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 commented on issue #3018: Fix the pre
aggregation is difference between old and new selector
Posted by GitBox <gi...@apache.org>.
EmmyMiao87 commented on issue #3018: Fix the pre aggregation is difference between old and new selector
URL: https://github.com/apache/incubator-doris/pull/3018#issuecomment-593753202
#2101
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #3018: Fix the pre
aggregation is difference between old and new selector
Posted by GitBox <gi...@apache.org>.
morningman merged pull request #3018: Fix the pre aggregation is difference between old and new selector
URL: https://github.com/apache/incubator-doris/pull/3018
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request
#3018: Fix the pre aggregation is difference between old and new selector
Posted by GitBox <gi...@apache.org>.
EmmyMiao87 commented on a change in pull request #3018: Fix the pre aggregation is difference between old and new selector
URL: https://github.com/apache/incubator-doris/pull/3018#discussion_r386195537
##########
File path: fe/src/main/java/org/apache/doris/planner/MaterializedViewSelector.java
##########
@@ -279,16 +280,30 @@ private void checkCompensatingPredicates(Set<String> columnsInPredicates,
* @param candidateIndexIdToSchema
*/
- private void checkGrouping(Set<String> columnsInGrouping, Map<Long, List<Column>> candidateIndexIdToSchema) {
+ private void checkGrouping(Set<String> columnsInGrouping, Map<Long, List<Column>> candidateIndexIdToSchema,
+ KeysType keysType) {
Iterator<Map.Entry<Long, List<Column>>> iterator = candidateIndexIdToSchema.entrySet().iterator();
while (iterator.hasNext()) {
Map.Entry<Long, List<Column>> entry = iterator.next();
Set<String> indexNonAggregatedColumnNames = new TreeSet<>(String.CASE_INSENSITIVE_ORDER);
List<Column> candidateIndexSchema = entry.getValue();
candidateIndexSchema.stream().filter(column -> !column.isAggregated())
.forEach(column -> indexNonAggregatedColumnNames.add(column.getName()));
- // When the candidate index is SPJ type, it passes the verification directly
- if (indexNonAggregatedColumnNames.size() == candidateIndexSchema.size()) {
+ /*
+ If there is no aggregated column in duplicate table, the index will be SPJ.
+ For example:
+ duplicate table (k1, k2, v1)
+ mv index (k1, v1)
+ When the candidate index is SPJ type, it passes the verification directly
+
+ If there is no aggregated column in aggregate index, the index will be deduplicate table.
+ For example:
+ aggregate table (k1, k2, v1 sum)
+ mv index (k1, k2)
+ This kind of index is SPJG which same as select k1, k2 from aggregate_table group by k1, k2.
+ It also need to check the grouping column using following steps.
+ */
Review comment:
OK
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#3018: Fix the pre aggregation is difference between old and new selector
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #3018: Fix the pre aggregation is difference between old and new selector
URL: https://github.com/apache/incubator-doris/pull/3018#discussion_r386031136
##########
File path: fe/src/main/java/org/apache/doris/planner/MaterializedViewSelector.java
##########
@@ -279,16 +280,30 @@ private void checkCompensatingPredicates(Set<String> columnsInPredicates,
* @param candidateIndexIdToSchema
*/
- private void checkGrouping(Set<String> columnsInGrouping, Map<Long, List<Column>> candidateIndexIdToSchema) {
+ private void checkGrouping(Set<String> columnsInGrouping, Map<Long, List<Column>> candidateIndexIdToSchema,
+ KeysType keysType) {
Iterator<Map.Entry<Long, List<Column>>> iterator = candidateIndexIdToSchema.entrySet().iterator();
while (iterator.hasNext()) {
Map.Entry<Long, List<Column>> entry = iterator.next();
Set<String> indexNonAggregatedColumnNames = new TreeSet<>(String.CASE_INSENSITIVE_ORDER);
List<Column> candidateIndexSchema = entry.getValue();
candidateIndexSchema.stream().filter(column -> !column.isAggregated())
.forEach(column -> indexNonAggregatedColumnNames.add(column.getName()));
- // When the candidate index is SPJ type, it passes the verification directly
- if (indexNonAggregatedColumnNames.size() == candidateIndexSchema.size()) {
+ /*
+ If there is no aggregated column in duplicate table, the index will be SPJ.
+ For example:
+ duplicate table (k1, k2, v1)
+ mv index (k1, v1)
+ When the candidate index is SPJ type, it passes the verification directly
+
+ If there is no aggregated column in aggregate index, the index will be deduplicate table.
+ For example:
+ aggregate table (k1, k2, v1 sum)
+ mv index (k1, k2)
+ This kind of index is SPJG which same as select k1, k2 from aggregate_table group by k1, k2.
+ It also need to check the grouping column using following steps.
+ */
Review comment:
Please add a UT to cover this case.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org