[incubator-doris] 09/10: [BUG] ReAgg when adding agg mv on dup base table (#4587)

[mo...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch branch-0.13
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git

commit 3f4110e71b46a57e15fc6792b110d7392270ecb7
Author: EmmyMiao87 <52...@qq.com>
AuthorDate: Sun Sep 13 19:17:35 2020 +0800

    [BUG] ReAgg when adding agg mv on dup base table (#4587)
    
    When the keystype of mv and base table is difference, Doris should execute
    sorting schema change instead of linked schema change.
    If doesn't, the data size of mv actually is same as base table.
    This will cause mv to have no pre-aggregation effect at all.
    The query will not choose mv.
    
    This commit fixed this problem. Fixed #4586
---
 be/src/olap/schema_change.cpp | 27 +++++++++++++++++++++++++--
 1 file changed, 25 insertions(+), 2 deletions(-)

diff --git a/be/src/olap/schema_change.cpp b/be/src/olap/schema_change.cpp
index 3522b1b..81ace25 100644
--- a/be/src/olap/schema_change.cpp
+++ b/be/src/olap/schema_change.cpp
@@ -2090,14 +2090,37 @@ OLAPStatus SchemaChangeHandler::_parse_request(TabletSharedPtr base_tablet,
         }
     }
 
+    const TabletSchema& ref_tablet_schema = base_tablet->tablet_schema();
+    const TabletSchema& new_tablet_schema = new_tablet->tablet_schema();
+    if (ref_tablet_schema.keys_type() != new_tablet_schema.keys_type()) {
+        // only when base table is dup and mv is agg
+        // the rollup job must be reagg.
+        *sc_sorting = true;
+        return OLAP_SUCCESS; 
+    }
+
+    // If the sort of key has not been changed but the new keys num is less then base's,
+    // the new table should be re agg.
+    // So we also need to set  sc_sorting = true.
+    // A, B, C are keys(sort keys), D is value
+    // followings need resort:
+    //      old keys:    A   B   C   D
+    //      new keys:    A   B
+    if (new_tablet_schema.keys_type() != KeysType::DUP_KEYS
+            && new_tablet->num_key_columns() < base_tablet->num_key_columns()) {
+        // this is a table with aggregate key type, and num of key columns in new schema
+        // is less, which means the data in new tablet should be more aggregated.
+        // so we use sorting schema change to sort and merge the data.
+        *sc_sorting = true;
+        return OLAP_SUCCESS;
+    }
+
     if (base_tablet->num_short_key_columns() != new_tablet->num_short_key_columns()) {
         // the number of short_keys changed, can't do linked schema change
         *sc_directly = true;
         return OLAP_SUCCESS;
     }
 
-    const TabletSchema& ref_tablet_schema = base_tablet->tablet_schema();
-    const TabletSchema& new_tablet_schema = new_tablet->tablet_schema();
     for (size_t i = 0; i < new_tablet->num_columns(); ++i) {
         ColumnMapping* column_mapping = rb_changer->get_mutable_column_mapping(i);
         if (column_mapping->ref_column < 0) {


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org