You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kylin.apache.org by "XIE FAN (JIRA)" <ji...@apache.org> on 2016/11/21 02:23:58 UTC

[jira] [Created] (KYLIN-2217) Reducers build dictionaries locally

XIE FAN created KYLIN-2217:
------------------------------

             Summary: Reducers build dictionaries locally
                 Key: KYLIN-2217
                 URL: https://issues.apache.org/jira/browse/KYLIN-2217
             Project: Kylin
          Issue Type: Improvement
    Affects Versions: v1.5.4.1
            Reporter: XIE FAN
             Fix For: Future


In KYLIN-1851, we reduce the peek memory usage of the dictionary-building procedure by splitting a single Trie tree structure to Trie forest. But there still exist a bottleneck that all the dictionaries are built in Kylin client. In this issue, we want to use multi reducers to build different dictionaries locally and concurrently，which can further reduce the peek memory usage as well as speed up the dictionary-building procedure.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)