[jira] [Created] (KYLIN-3920) Don't merge same dictionaries when merge dictionary

["Yuzhang QIU (JIRA)" <ji...@apache.org>]

Yuzhang QIU created KYLIN-3920:
----------------------------------

             Summary: Don't merge same dictionaries when merge dictionary
                 Key: KYLIN-3920
                 URL: https://issues.apache.org/jira/browse/KYLIN-3920
             Project: Kylin
          Issue Type: Improvement
          Components: Others
    Affects Versions: v2.5.2
            Reporter: Yuzhang QIU


Hi team:
   I found DictionaryManager will pass some dictionaries to DictionaryGenerator to merge them when there is different one among them. But If there are 3 dictionaries {Dic1, Dic1, Dic2} in 3 segments, kylin may don't need to merge Dic1 and Dic1, which won't add same value into new dictionary twice.
  If I misunderstand the merge job logic, please feel free to correct me!
  Here is the code snapshot at DictionaryManager.java:251

```
        boolean identicalSourceDicts = true;
        for (int i = 1; i < dicts.size(); ++i) {
            if (!dicts.get(0).getDictionaryObject().equals(dicts.get(i).getDictionaryObject())) {
                identicalSourceDicts = false;
                break;
            }
        }

        if (identicalSourceDicts) {
            logger.info("Use one of the merging dictionaries directly");
            return dicts.get(0);
        } else {
            Dictionary<String> newDict = DictionaryGenerator.mergeDictionaries(DataType.getType(newDictInfo.getDataType()), dicts);
            return trySaveNewDict(newDict, newDictInfo);
        }
```

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)