You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kylin.apache.org by "kangkaisen (JIRA)" <ji...@apache.org> on 2017/03/13 08:38:04 UTC

[jira] [Created] (KYLIN-2506) Refactor Global Dictionary

kangkaisen created KYLIN-2506:
---------------------------------

             Summary: Refactor Global Dictionary
                 Key: KYLIN-2506
                 URL: https://issues.apache.org/jira/browse/KYLIN-2506
             Project: Kylin
          Issue Type: Improvement
          Components: General
    Affects Versions: v2.0.0
            Reporter: kangkaisen
            Assignee: kangkaisen
             Fix For: v2.0.0


The main points of this refactor:
1 Fix the bug that the RemoveListener of LoadingCache swallowed any exceptions when building the GlobalDict.
2 Fix the bug that the HDFS filename of DictSliceKey had Illegal characters.
3 Fix the bug that the HDFS filename of DictSliceKey maybe longer than 255.
4 Fix the bug that DictNode split failed if value length greater than 255 bytes.
5 Decouple the build and query of GlobalDict: 
Abstract the builder of AppendTrieDictionary to AppendTrieDictionaryBuilder; Add LoadingCache to AppendTrieDictionary and make AppendTrieDictionary is only readable.
6 Remove dependence of LoadingCache when building the GlobalDict.
7 Abstract the HDFS operations to GlobalDictStore.
8 Abstract the metadata of GlobalDict to GlobalDictMetadata.
9 Delete CachedTreeMap.
10 Remove the support of multithreading concurrent build and I will add distributed lock for GlobalDict later.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)