[jira] [Commented] (KYLIN-2506) Refactor Global Dictionary

["Billy Liu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KYLIN-2506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15922682#comment-15922682 ] 

Billy Liu commented on KYLIN-2506:
----------------------------------

+1. 

Will this refactor include the GUI part? Currently, user has to define GlobalDictionary manually, even he has defined the precise distinct count measure. Could we merge them into one step?

> Refactor Global Dictionary
> --------------------------
>
>                 Key: KYLIN-2506
>                 URL: https://issues.apache.org/jira/browse/KYLIN-2506
>             Project: Kylin
>          Issue Type: Improvement
>          Components: General
>    Affects Versions: v2.0.0
>            Reporter: kangkaisen
>            Assignee: kangkaisen
>             Fix For: v2.0.0
>
>
> The main points of this refactor:
> 1 Fix the bug that the RemoveListener of LoadingCache swallowed any exceptions when building the GlobalDict.
> 2 Fix the bug that the HDFS filename of DictSliceKey had Illegal characters.
> 3 Fix the bug that the HDFS filename of DictSliceKey maybe longer than 255.
> 4 Fix the bug that DictNode split failed if value length greater than 255 bytes.
> 5 Decouple the build and query of GlobalDict: 
> Abstract the builder of AppendTrieDictionary to AppendTrieDictionaryBuilder; Add LoadingCache to AppendTrieDictionary and make AppendTrieDictionary is only readable.
> 6 Remove dependence of LoadingCache when building the GlobalDict.
> 7 Abstract the HDFS operations to GlobalDictStore.
> 8 Abstract the metadata of GlobalDict to GlobalDictMetadata.
> 9 Delete CachedTreeMap.
> 10 Remove the support of multithreading concurrent build and I will add distributed lock for GlobalDict later.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)