You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "kangkaisen (JIRA)" <ji...@apache.org> on 2017/04/14 06:16:41 UTC
[jira] [Comment Edited] (KYLIN-2506) Refactor Global Dictionary
[ https://issues.apache.org/jira/browse/KYLIN-2506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15967434#comment-15967434 ]
kangkaisen edited comment on KYLIN-2506 at 4/14/17 6:15 AM:
------------------------------------------------------------
KYLIN-2506 Refactor Global Dictionary https://github.com/apache/kylin/commit/ce8b24f60a01d03eb57a223dbae095e7ceb98c7f
KYLIN-2506 Refactor ZookeeperDistributedJobLock https://github.com/apache/kylin/commit/6416df157c0ac2f197f69a613bbc82e8e60fc37f
KYLIN-2506 Add distributed lock for GlobalDictionaryBuilder https://github.com/apache/kylin/commit/0018fafc7005f9b72dfff9ba4ae53054e3ff63ad
These three commits have passed the UT and IT
was (Author: kangkaisen):
KYLIN-2506 Refactor Global Dictionary https://github.com/apache/kylin/commit/ce8b24f60a01d03eb57a223dbae095e7ceb98c7f
KYLIN-2506 Refactor ZookeeperDistributedJobLock https://github.com/apache/kylin/commit/fd3b7655839af8dcd2370570ed9e426db410eb12
KYLIN-2506 Add distributed lock for GlobalDictionaryBuilder https://github.com/apache/kylin/commit/fc01989c2fff440a98822186beac6383902c1e0b
These three commits have passed the UT and IT
> Refactor Global Dictionary
> --------------------------
>
> Key: KYLIN-2506
> URL: https://issues.apache.org/jira/browse/KYLIN-2506
> Project: Kylin
> Issue Type: Improvement
> Components: General
> Affects Versions: v2.0.0
> Reporter: kangkaisen
> Assignee: kangkaisen
> Fix For: v2.0.0
>
>
> The main points of this refactor:
> 1 Fix the bug that the RemoveListener of LoadingCache swallowed any exceptions when building the GlobalDict.
> 2 Fix the bug that the HDFS filename of DictSliceKey had Illegal characters.
> 3 Fix the bug that the HDFS filename of DictSliceKey maybe longer than 255.
> 4 Fix the bug that DictNode split failed if value length greater than 255 bytes.
> 5 Decouple the build and query of GlobalDict:
> Abstract the builder of AppendTrieDictionary to AppendTrieDictionaryBuilder; Add LoadingCache to AppendTrieDictionary and make AppendTrieDictionary is only readable.
> 6 Remove dependence of LoadingCache when building the GlobalDict.
> 7 Abstract the HDFS operations to GlobalDictStore.
> 8 Abstract the metadata of GlobalDict to GlobalDictMetadata.
> 9 Delete CachedTreeMap.
> 10 Add distributed lock for GlobalDict.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)